**HABITUATION MECHANISMS AND THEIR IMPACT ON COGNITIVE FUNCTION**

**Topic Editors Susanne Schmid, Donald A. Wilson and Catharine H. Rankin** 

INTEGRATIVE NEUROSCIENCE

#### *FRONTIERS COPYRIGHT STATEMENT*

© Copyright 2007-2015 Frontiers Media SA. All rights reserved.

All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.

Cover image provided by Ibbl sarl, Lausanne CH

**ISSN** 1664-8714 **ISBN** 978-2-88919-462-9 **DOI** 10.3389/978-2-88919-462-9

# *ABOUT FRONTIERS*

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

# *FRONTIERS JOURNAL SERIES*

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing.

All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

# *DEDICATION TO QUALITY*

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view.

By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

# *WHAT ARE FRONTIERS RESEARCH TOPICS?*

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area!

Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **HABITUATION MECHANISMS AND THEIR IMPACT ON COGNITIVE FUNCTION**

Topic Editors:

**Susanne Schmid,** University of Western Ontario, Canada **Donald A. Wilson,** New York University, USA **Catharine H. Rankin,** University of British Columbia, Canada

The dual process theory of habituation. From: Groves & Thompson, Psychological Review, 1970, 77:419-450

Habituation describes the progressive decrease of the amplitude or frequency of a motor response to repeated sensory stimulation that is not caused by sensory receptor adaptation or motor fatigue. Habituation can occur in different time scales: habituation within a testing session has been termed short-term habituation, whereas habituation across testing sessions has been termed long-term habituation. Generally, the more spaced the stimuli for inducing habituation are presented (i.e. the slower habituation is induced), the longer it seems to take to recover the behavioural response to its initial magnitude. Habituation is opposed by behavioural sensitization, which is thought to be an independent mechanism that leads to an increased

behavioural response, especially if the sensory stimulus is annoying or aversive.

Habituation provides an important mechanism for filtering sensory information, as it allows for filtering out irrelevant stimuli and thereby focussing on important stimuli, a prerequisite for many cognitive tasks. The importance of habituation is demonstrated in people with mental disorders that are associated with disruptions in habituation, e.g. schizophrenia and autism spectrum disorders. The inability to filter out irrelevant information in patients with these disorders strongly correlates with disruptions in higher cognitive function, such as in different types of memory and attention.

Habituation is also considered to be the most basic form of non-associative implicit learning, and it can be observed throughout the animal kingdom. Based on the importance of habituation for cognitive function and therefore for the survival of an animal, it is assumed that habituation mechanisms are highly conserved across species. On the other hand, there is emerging evidence for a multitude of homo- and heterosynaptic mechanisms underlying habituation, depending on the modality of sensory stimulation, the level of sensory information processing where habituation occurs, and the temporal composition of sensory stimulation.

The scope of this Frontier Research Topic is to give an overview over the concept of habituation, different animal and behavioural models used for studying habituation mechanisms, as well as the different synaptic and molecular processes suggested to play a role in behavioural habituation through Original Research Articles, Methods, Hypothesis & Theory Articles, and Reviews.

# Table of Contents

*05 Habituation Mechanisms and their Importance for Cognitive Function* Susanne Schmid, Donald A. Wilson and Catharine H. Rankin *07 Network Processes Involved in the Mediation of Short-Term Habituation in Aplysia: Contribution of Intrinsic Regulation of Excitability and Synaptic Augmentation* Thomas M. Fischer, Daniel A. Jacobson and Kristen Demorest-Hayes *19 Habituation of Reflexive and Motivated Behavior in Mice With Deficient BK Channel Function* Marei Typlt, Magdalena Mirkowski, Erin Azzopardi, Peter Ruth, Peter K. D. Pilz and Susanne Schmid *27 Longterm-Habituation of the Startle Response in Mice is Stimulus Modality, but not Context Specific* Peter K. D. Pilz, Stephan W. Arnold, Anja T. Rischawy and Claudia F. Plappert *37 Saliency Mapping in the Optic Tectum and its Relationship to Habituation* Arkadeb Dutta and Yoram Gutfreund *50 Adaptation in the Auditory System: An Overview* David Pérez-González and Manuel S. Malmierca *60 Stressors Impair Odor Recognition Memory Via an Olfactory Bulb-Dependent Noradrenergic Mechanism* Laura C. Manella, Samuel Alperin and Christiane Linster *72 Habituation of the Responsiveness of Mesolimbic and Mesocortical Dopamine Transmission to Taste Stimuli* Maria A. De Luca *77 Habituation of Reinforcer Effectiveness* David R. Lloyd, Douglas J. Medina, Larry W. Hawk, Whitney D. Fosco and Jerry B. Richards *97 The Mechanism of Dishabituation* Genevieve Z. Steiner and Robert J. Barry *105 Habituation, Sensitization, and Pavlovian Conditioning* Münire Özlem Çevik

#### *Susanne Schmid1 \*, Donald A. Wilson2,3,4 and Catharine H. Rankin5,6*

*<sup>1</sup> Anatomy and Cell Biology, University of Western Ontario, London, ON, Canada*

*<sup>2</sup> Emotional Brain Institute, Nathan Kline Institute for Psychiatric Research, Orangeburg, NY, USA*

*<sup>3</sup> Department of Physiology and Neuroscience, New York University Medical Center, New York, NY, USA*

*<sup>4</sup> Department of Child and Adolescent Psychiatry, New York University Medical Center, New York, NY, USA*


*\*Correspondence: susanne.schmid@schulich.uwo.ca*

#### *Edited by:*

*Sidney A. Simon, Duke University, USA*

#### *Reviewed by:*

*Adriano B. L. Tort, Federal University of Rio Grande do Norte, Brazil*

#### **Keywords: habituation, sensorimotor gating, sensory filtering, synaptic mechanisms, behavioral plasticity, learning and memory**

How does our brain form and store memories? One way to approach this mystery is to study a very basic form of learning habituation—in a relatively simple nervous system. Habituation describes the progressive decrease of the amplitude or frequency of a motor response to repeated sensory stimulation that is not caused by sensory receptor adaptation or motor fatigue. A multitude of different organisms, behaviors, and experimental approaches have been used to study habituation, but still surprisingly little is known about the underlying mechanisms. A theoretical framework of the concept of habituation has been laid by Thompson and Spencer (1966), and by the dual process theory of Groves and Thompson (1970), which describes habituation and sensitization as two independent processes that interact to yield the final behavioral outcome. In a symposium in 2009, the original concept was revisited and the definitions of habituation (and dishabituation) were slightly revised for clarity; however, remarkably there were only few changes to the defining characteristics (Rankin et al., 2009). It is becoming evident that behavioral habituation is caused by different mechanisms depending on time frame of stimulation, type of sensory pathway studied, and hierarchical level of signal processing. On the other hand, habituation mechanisms seem to be highly conserved, underlining the importance of habituation for the survival of a species (see Schmid et al., 2010). The scope of this Frontiers Research Topic is to give an overview over the concept of habituation, the different animal and behavioral models used for studying habituation mechanisms, as well as the different synaptic and molecular processes suggested to play a role in behavioral habituation.

Fischer et al. (2014) studied short-term habituation of the gill-withdrawal reflex in Aplysia. In accordance with the notion of different mechanisms mediating habituation in different time frames and different pathways, they report an intrinsic mechanism that is specific for short-term habituation at short training intervals of 1s. Typlt et al. (2013b) investigated the role of a voltage-and calcium activated potassium channel (BK channel) in short- and long-term habituation of an elicited behavior (acoustic startle) versus a motivated exploratory behavior using transgenic mice, and further confirm disparate, yet evolutionary highly conserved habituation mechanisms. Pilz et al. (2014) tackled a contentious issue of whether long-term habituation of acoustic startle in mice is context specific. They report that long-term habituation is stimulus-modality specific, but not context specific, confirming it as a non-associative form of learning. Dutta and Gutfreund (2014) review data from barn owls and primates on computation of saliency in the optic tectum/superior colliculus and how this is linked to habituation and neural adaptation. Perez-Gonzalez and Malmierca (2014) review different forms of spike adaptation in auditory neurons of different levels of auditory processing hierarchy. These mechanisms lead to sensory filtering and habituation of perception. Manella et al. (2013) studied how the modulatory norepinephrine system in the brain influences odor habituation and odor memory in rats.

Besides the importance of understanding the underlying mechanisms of habituation as a basic form of learning or sensory filtering, some articles go beyond understanding mechanisms of habituation and explore how its disruption impacts other cognitive domains and higher cognitive function. Typlt et al. (2013a) link habituation deficits to impairments in spatial learning. The Mini Review of De Luca (2014) sheds light on the mechanism of the habituation phenomenon of mesolimbic and mesocortical dopamine transmission in response to taste stimuli, and its putative role as a marker of cortical dysfunction in specific conditions such as addiction. Related to this topic, Lloyd et al. (2014) review the habituation of reinforcer effectiveness and the role of dopamine neurotransmission in habituation to the reinforcer. They indicate that behavioral disorders such as obesity or attention deficit hyperactivity disorder (ADHD) may be caused by abnormal habituation to the reinforcer due to genetic or environmental factors.

Interestingly, studying the electrodermal orienting reflex in humans, Steiner and Barry (2014) argue against the dualprocess theory's explanation that dishabituation is caused by sensitization, and instead suggest that dishabituation is a disruption of the habituation process, with its magnitude determined by the corresponding arousal level. It is certainly debatable to what extent this can be generalized to other modalities and pathways. In a theoretical essay Cevik (2014) argues that the impact of a stimulus on behavior and its potential to modulate the effects of other stimuli increase as its distance from the body decreases, an interesting and certainly also debatable concept.

In summary, this research topic contains original research articles, reviews, and theoretical essays that provide an updated view on different models for studying habituation, its underlying mechanisms, and its importance as prerequisite for higher cognitive function. The number and high quality of the papers on this topic provide support for the notion that habituation is a rich area of study, touching on a number of important questions related to behavioral plasticity.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 02 December 2014; accepted: 16 December 2014; published online: 08 January 2015.*

*Citation: Schmid S, Wilson DA and Rankin CH (2015) Habituation mechanisms and their importance for cognitive function. Front. Integr. Neurosci. 8:97. doi: 10.3389/ fnint.2014.00097*

*This article was submitted to the journal Frontiers in Integrative Neuroscience.*

*Copyright © 2015 Schmid, Wilson and Rankin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Network processes involved in the mediation of short-term habituation in Aplysia: contribution of intrinsic regulation of excitability and synaptic augmentation

# *Thomas M. Fischer\*, Daniel A. Jacobson and Kristin Demorest-Hayes*

*Department of Psychology, Wayne State University, Detroit, MI, USA*

#### *Edited by:*

*Catharine H. Rankin, University of British Columbia, Canada*

#### *Reviewed by:*

*Donald A. Wilson, New York University School of Medicine, USA William G. Wright, Schmid College of Science, USA*

#### *\*Correspondence:*

*Thomas M. Fischer, Department of Psychology, Wayne State University, 5057 Woodward Avenue, 7th Floor, Detroit, MI 48202, USA e-mail: thomas.fischer@wayne.edu*

Short-term habituation (STH) is the decrease in behavioral responding observed during repeated stimulation at regular intervals. For siphon-elicited siphon withdrawal in *Aplysia* (S-SWR), we previously showed that the amplitude of responses measured in LFS-type siphon motor neurons (LFS MNs) during training is dependent on the stimulus interval used and is training-site specific. The major source of excitation from siphon stimulation onto the LFS MNs comes from the L29 interneurons. Here we examined the role of the L29s in STH by addressing two questions:

(1) What are the relative contributions of intrinsic regulation of excitability and network inhibition on L29 activity during STH training?

By activating L29s with intracellular current injection, we found that intrinsic changes in excitability occur, but only at short training intervals (1 s). We also demonstrated that network inhibition is not required for regulating L29 responses during training, indicating that any expression of inhibition is redundant to the excitability changes.

(2) How does L29 synaptic plasticity contribute to the maintenance of training sitespecificity exhibited in LFS MNs?

When training stimuli are delivered 1 s apart [1 s, interstimulus interval (ISI)], L29 responses decrease in both stimulated (trained) and un-stimulated (untrained) pathways, yet site-specificity of training is maintained in the LFS MNs. Our results suggest that activity-dependent synaptic facilitation (augmentation; AUG) expressed by the L29s acts to compensate for the decreased activity in the untrained pathway. First, we demonstrated that the L29-LFS synapse exhibits significant AUG with L29 activation ata1s ISI. Second, we showed that the induction of AUG prevents the reduction in siphon-evoked LFS responses that is otherwise observed with decreased L29 activity. Collectively, our results support a role for the L29s in regulating network dynamics during STH training, but only at rapid (1 s ISI) training intervals.

**Keywords: interneuron, excitability, inhibition, reflex, mechanoreceptor, network**

#### **INTRODUCTION**

Dynamic processes within neural networks underlie the capacity for organisms to utilize information to adaptively regulate behavior. These processes include intrinsic changes resulting from activity, and/or extrinsic regulation such as synaptic inhibition or neuromodulation. Further, these processes often occur concurrently at multiple sites in a network (Frost et al., 1988; Chandler and Grossberg, 2012; Garrido et al., 2013). This inherent complexity of neural networks dictates that any understanding of neural mechanisms underlying behavioral plasticity must not only catalog the underlying cellular changes, but also describe how these processes interact to yield a net behavioral output. We have examined short-term habituation (STH, also referred to as "within-session" habituation: Thompson and Spencer, 1966; Thompson, 2009) as a means to understand how a simple neural network dynamically adjusts behavioral responsiveness to match sensory information from the environment. STH refers to the decrement in responding observed during training with regularly

spaced stimuli, with the rate of decrement determined by the interval between stimuli (ISI). We utilize the siphon withdrawal response (SWR) in the marine mollusk *Aplysia californica* as an experimental model system, which has proven useful in relating cellular processes directly to diverse forms of behavioral regulation (Carew and Kandel, 1973; Wright and Carew, 1995; Cohen et al., 1997; Fischer et al., 1997; Sutton et al., 2001; Philips et al., 2011). In this preparation, the effects of habituation training are restricted to the site of training, even when a second"untrained" site resides a few centimeters apart on the siphon. This site-specificity of training is taken as evidence that sensory afferents represent the primary locus of change with habituation (Frost et al., 1997; Ezzeddine and Glanzman, 2003).

Siphon MNs receive input both directly from sensory neurons as well as from a restricted set of identified interneurons such as the L29-type excitatory interneurons (**Figure 1B**; Hawkins et al., 1981; Cleary et al., 1995; Frost and Kandel, 1995). Depending on the stimulus, interneurons are estimated to provide around 75%

of the net evoked input to siphon MNs (Trudeau and Castellucci, 1992), with single L29 neurons (out of five total) accounting for an average of 15% of this input (Fischer and Carew, 1993). Two parallel sensory pathways can be distinguished based on their stimulus thresholds (Byrne et al., 1978b; Frost et al., 1997; Illich andWalters, 1997; Calin-Jageman and Fischer, 2007). The well-characterized LE mechanoreceptors have cell bodies within the abdominal ganglion (Castellucci et al., 1970), and have stimulus activation thresholds higher than that required to evoke responses in MNs (Frost et al., 1997; Illich andWalters, 1997; Calin-Jageman and Fischer, 2007). The somata of the lower-threshold sensory neurons have yet to be identified, hence we refer to these as the unidentified low threshold (ULT) mechanoreceptors (Calin-Jageman and Fischer, 2007; Fischer et al., 2011).

Our studies on STH have used low-intensity siphon taps that only activate the ULTs. We found that changes in the ULTs alone can account for SWR network responses during habituation training at a 30 s ISI, an interval commonly used in studies of habituation in this preparation (Castellucci et al., 1970; Pinsker et al., 1970; Carew et al., 1972; Peretz et al., 1976; Castellucci et al., 1978; Rankin and Carew, 1987; Falk et al., 1993; Ezzeddine and Glanzman, 2003; Fischer et al., 2011). First, STH-induced regulation of both the L29 and LFS MN activity was training site-specific (Fischer et al., 2011), implicating sensory input as the key site of change (Frost et al., 1997; Ezzeddine and Glanzman, 2003). Second, there is a direct linear relationship between siphon-evoked ULT activity and responses in both L29 interneurons and LFS-type siphon MNs with single (non-habituating) stimuli. During habituation training, the ratio of ULT to LFS activity remained constant as overall network activity adjusted to a reduced asymptotic level. Together, these observations indicate that changes in the activity level of the ULTs alone can account for the observed changes in the network.

Two observations indicated that additional mechanisms beyond the regulation of ULT activity were involved in network regulation at a more rapid training interval (1 s ISI). First, the rate of decrement during training was faster for both the L29s and LFS MNs than that of ULT activity. Consistent with this, the ratio

of ULT activity to LFS responses during training was non-linear, indicating the involvement of processes in addition to decreased sensory activity. Second, site-specificity of training was not preserved in the L29s, which exhibited a significant decrease in activity in response to stimulation of an untrained pathway. Despite this, site-specificity of training was preserved in the LFS MNs (Fischer et al., 2011). This raises the question as to how evoked MN activity remains constant when the activity of a major source of excitation is reduced.

Here, we focused primarily on the role of the L29 interneurons in regulating SWR network activity at rapid (1 s ISI) training intervals. L29s are of particular interest both because they are a major source of excitatory input to LFS MNs (Fischer and Carew, 1993; Frost and Kandel, 1995) and they are a principal target for the L30 inhibitory interneurons that regulate the SWR following tactile stimulation (**Figure 1B**, Hawkins et al., 1981; Fischer and Carew, 1995; Frost and Kandel, 1995; Calin-Jageman and Fischer, 2003a). First, we examined whether L29 excitability (or intrinsic plasticity: Zhang and Linden, 2003) is regulated with STH. We describe excitability changes at a 1 s ISI that are absent at a 30 s ISI, and suggest that this form of plasticity could account for both the accelerated rate of change observed in the network ata1sISI as well as the lack of site-specificity of training observed in the L29s. Second, we explored the hypothesis that augmentation (AUG), a form of short-term synaptic enhancement expressed by the L29s, can act to maintain site-specificity of training. We show that training at a 1 s ISI induces AUG in the L29s, and that the induction of AUG can compensate for reduced L29 activity. Collectively, our results help confirm that that regulation of ULT activity is a primary underlying mechanism of STH at a 30 s ISI, with additional processes intrinsic to the L29s contributing ata1s ISI.

#### **MATERIALS AND METHODS ANIMALS**

Wild-caught adult *A. Californica* (150–400 g) were acquired from Marinus Incorporated (Long Beach, CA, USA). Animals were housed in a custom 600-L aquarium with circulating Instant Ocean artificial seawater (ASW; Aquarium Systems, Mentor, OH,

site-specificity of training, as illustrated. **(B)** Simplified version of the SWR network emphasizing the major elements in the reflex circuit. While only one cell of each type is shown, cells are multiply represented in the network. Two parallel sensory pathways from the siphon provide excitatory input, the lower threshold ULTs and the higher threshold LEs. Figure modified from Fischer et al. (2011).

USA),maintained on a 12-h light:dark cycle, and fed dried seaweed (nori) five times per week.

#### **EXPERIMENTAL PREPARATION**

Experiments were conducted with semi-intact preparations consisting of the siphon, mantle shelf, siphon nerve, and abdominal ganglion (**Figure 1A**). Animals were anesthetized by injection of isotonic MgCl2 into the body cavity at a volume of 0.5 ml/g of body weight. The siphon and mantle, along with the siphon nerve and connected abdominal ganglion, were then dissected from the animal. The preparation was transferred to a two-chambered recording dish coated with Sylgard (Dow Corning, Midland, MI, USA), with the abdominal ganglion placed in a separate compartment from the rest of the preparation. The abdominal ganglion was pinned ventral side up using minute pins, and the mantle was secured dorsal side up using 26*<sup>G</sup>* hypodermic needles. The siphon was not restrained. Throughout the experiment, the siphon was continuously perfused with ASW through a cannula placed in the siphon artery. At least 45 min of post-dissection recovery time was allowed prior to physiological recordings.

### **INTRACELLULAR RECORDINGS**

To facilitate intracellular recordings, the sheath covering the left hemi-ganglion was surgically removed to expose the underlying neurons. Neurons were impaled with glass microelectrodes filled with 3 M KCL (resistance 6–12 m-). Potentials were amplified on a Dagan IX2-700 amplifier (Dagan, Minneapolis, MN, USA) and then digitized (Powerlab 8SP) for computer analysis using the Chart software package (ADInstruments, Colorado Springs, CO, USA). Individual cell types were identified using established criteria (Hawkins et al., 1981; Frost and Kandel, 1995). LFS MNs were identified based on their position within the abdominal ganglion, and by observing the characteristic siphon movements produced during their intracellular activation (Hickie and Walters, 1995). No distinction was made between LFS MN sub-types. L29 interneurons were identified based on their size and position, their recruitment of recurrent inhibition during intracellular activation, and their characteristic response to siphon tap (Hawkins et al., 1981; Fischer and Carew, 1993; Frost and Kandel, 1995). L29 interneurons were excluded from analysis if they responded with fewer than five action potentials to a siphon tap. This likely biases our sample towards the L29-A sub-type, which exhibit a greater siphon tap response than the L29-B sub-type (Fischer and Carew, 1995; Fang and Clark, 1996).

### **EXPERIMENTAL PROCEDURES**

In these experiments, training was accomplished either through tactile stimulation via siphon taps or through activating individual neurons directly with intracellular current injection. The siphon was tapped with glass probes attached to a stimulatordriven mechanical relay (**Figure 1A**; Fischer and Carew, 1993). The intensity of the tap (approximately 4 g/mm2) was similar to that we have used in previous experiments, which we have shown falls below the intensity threshold required to activate the LE sensory neurons when the siphon is not pinned to a substrate (Illich and Walters, 1997; Calin-Jageman and Fischer, 2007; Fischer et al., 2011). L29 neurons were allowed to remain at rest.

Siphon tap-evoked activity was measured by counting the number of spikes within a 500 ms period following the first evoked spike, as we have done previously (Fischer and Carew, 1995; Fischer et al., 2000; Fischer et al., 2011). Current was injected in the L29s through the same electrode used for recording, and consisted of 300 ms depolarizing square wave pulses, the approximate duration of the underlying depolarizing potential observed following siphon tap. The amplitude of the current was adjusted to produce 5–7 spikes, a number typically observed following siphon tap under our experimental conditions (Fischer et al., 2011). LFS MNs were hyperpolarized to −85 mV, approximately 40 mV below rest. At this holding potential, inhibitory synaptic potentials would be reversed, so measures of the tap-evoked complex PSP would consist of a combination of IPSPs and EPSPs. The complex PSP measure was obtained by calculating the area underneath the initial 500 ms of the siphon-evoked response (in μV ms) using Chart software package. This measurement accounts for changes in both response amplitude and duration of the complex PSP (Fischer and Carew, 1993). The amplitude of monosynaptic EPSPs of the L29 to LFS synapse was also measured using Chart.

To measure siphon tap-evoked ULT sensory neuron activity, we obtained extracellular recordings *en passant* from the siphon nerve with a monopolar suction electrode (A-M Systems, Carlsborg, WA, USA) using techniques described in detail elsewhere (Calin-Jageman and Fischer, 2007; Fischer et al., 2011). Briefly, the abdominal ganglion was placed under anesthesia with isotonic MgCl2 to isolate afferent activity (Hickie et al., 1997; Calin-Jageman and Fischer, 2007). A small portion of the nerve was aspirated into the electrode using negative pressure. Signals were amplified using a BioAmp CF, high-pass filtered at 10 Hz, low-pass filtered at 100 Hz, and then digitized (Powerlab 8SP) for computer analysis using the Chart software package (all from ADInstruments, Colorado Springs, CO, USA). As a quantitative measure, we first used Chart to compute the absolute value of the tap-evoked response measured from the nerve, which incorporates both positive and negative deflections of the complex evoked waveform (an example of this transform is shown in Fischer et al., 2011). We then determined the integral of the absolute value (in μV ms) for 300 ms following the onset of the evoked response, which is the approximate duration of the evoked response following a siphon tap.

In some experiments, we blocked inhibition within the SWR circuit through bath administration of 100 μm curare dissolved in ASW (D-tubocuranine; Sigma-Aldrich, St. Louis, MO, USA). Acetylcholine binding to nicotinic-like receptors is a major form of inhibitory neurotransmission in the central ganglia of *Aplysia* (Tauc and Gerschenfeld, 1962; Kehoe, 1972; Segal and Koester, 1982; Trudeau and Castellucci, 1993; Storozhuk and Castellucci, 1999), including from the L30 inhibitory interneurons (Calin-Jageman and Fischer, 2003a). Incubation with curare began at least 10 min before data collection and continued throughout the experiment. The effectiveness of administration was confirmed by observing the general increase in excitability of neurons with the blockage of inhibition (Trudeau and Castellucci, 1993; Lieb and Frost, 1997; Calin-Jageman and Fischer, 2003a).

Our standard experiment consisted of a (1) baseline measure; (2) training through repeated taps or current injections; and (3) a recovery measure. Training always began 5 min following the baseline measure, and the recovery measure was always obtained 5 min post-training. A 5 min interval was used based on previous observations that this interval does not result in a decrease of tap-evoked afferent activity across five consecutive stimuli (Calin-Jageman and Fischer, 2007). Training consisted of 30 taps or current injections delivered at either a 1 or 30 s ISI.

#### **DATA ANALYSIS**

Summary data are presented as means ± SEM; probability values reported are two-tailed. Statistical analysis was performed using the program GraphPad Prism (GraphPad Software Inc., San Diego, CA, USA). Analysis of site-specificity of training (**Figure 2**) was performed using two-way repeated measures ANOVA followed by Bonferroni post-tests to compare baseline, test, and recovery trials to each other. To analyze data obtained during training (**Figure 3**), we first determined whether the data for each training curve was best fit by a straight line or a one-phase exponential decay equation using an extra sum-of-squares *F* test. These comparisons took as the null hypothesis that the training curve would be best fit by a straight line, so a significant result indicates that a one-phase exponential decay provides a better fit for the data. Since our previous work determined that training using siphon tap always resulted in a non-linear, one-phase exponential decay (Fischer et al., 2011), this provides a means to assess whether other training methods (e.g., by current injection and/or training in curare) resulted in a different decay function. We determined whether training had an effect by comparing the first and 30th stimulus of a training session with a paired *t*-test. ULT and LFS MN recovery data (**Figure 4**) and L29 augmentation data (**Figures 5** and **6**) was assessed using a one-way ANOVA with the Bonferroni Multiple Comparison Test used for post-hoc comparison of baseline, test, and recovery measures.

# **RESULTS**

#### **ACTIVITY-DEPENDENT REGULATION OF L29 EXCITABILITY**

In our previous experiments, we found that site-specificity of STH depended upon the neuron-type examined and the training interval used. In particular, L29 excitatory interneurons exhibited generalization of training to untrained stimulus sites at a 1 s ISI but siphon-evoked responses remained training site-specific at a 30 s ISI (Fischer et al., 2011). Here, we examine the potential contribution of synaptic inhibition and intrinsic regulation of excitability to these interval-dependent differences in regulation of the L29s. We

for training at a 1 s ISI (TRAINED TEST), the second to test L29 responses 1 s following training at an untrained site (UNTRAINED TEST). Both the trained

significant decrease from baseline (\**p* < 0.05 from baseline using Bonferroni post tests).

**FIGURE 3 | L29 responses during habituation training with either siphon taps or injected current. (A)** Comparison of L29 responses during training with either siphon tap or repeated current injection. Data are expressed as a difference score from the first stimulus. At a 1 s ISI (left), a significant reduction is observed in both conditions, with a

following training, with no difference from baseline observed at 45 or 60 s following training. Conversely, LFS MN responses (right) remained significantly reduced from baseline across all time points tested. (\**p* < 0.05 compared to baseline using Bonferroni post tests). Subsequent recovery measures were not different from baseline across all conditions tested.

first examined whether the generalization of training would persist in the presence of 100 μm curare, which blocks synaptic inhibition within the SWR network (Trudeau and Castellucci, 1993) including from the L30 inhibitory interneurons (Calin-Jageman and Fischer, 2003a). We used the same procedure as in previous work (Fischer et al., 2011), which was adapted from Ezzeddine and Glanzman (2003). Two tappers were positioned on the siphon (**Figure 1A**); one tapper is used for habituation training ata1s ISI (trained pathway), the other used to assess site-specificity of training (untrained pathway). A representative example of an experiment preformed in curare is illustrated in **Figure 2A**. Curare was perfused into the recording chamber ten min prior

to the start of the experiment. To determine whether curare was effective, we took an initial measure prior to perfusion and compared this to our baseline measure from the "trained" tapper (data not shown). Consistent with previous observations (Trudeau and Castellucci, 1993; Calin-Jageman and Fischer, 2003a), we observed a significant increase of L29 tap-evoked responses in curare (mean difference = 8.0 spikes; *t*-test: *p* < 0.05). In the presence of curare, training still reduced responses in the L29s. To determine whether the effects of training were site-specific, a tap was delivered to the second tapper (untrained) as the 31st tap within the same ISI. As in our previous experiments, a reduction in the tap-evoked response was observed. Recovery from training was observed 5 min later.

the L29 to LFS monosynaptic EPSP in the siphon tap training condition. The

A summary of six experiments is provided in **Figure 2B**. A two-way ANOVA of all L29 activity data revealed a significant main effect of trial (*p* < 0.01), but no effect of training site (*p* = 0.50), indicating similar changes in both pathways. Bonferroni post-tests revealed a significant difference between baseline and test responses for both pathways (trained: *t* = 4.89, *p* < 0.01; untrained: *t* = 3.35, *p* < 0.01). Recovery measures

were significantly different from Test responses (trained: *t* = 5.02, *p* < 0.01; untrained: *t* = 3.03, *p* < 0.05). These data demonstrate that the generalization of habituation observed in the untrained pathway does not require synaptic inhibition.

different from baseline. (\**p* < 0.05 from baseline using Bonferroni post tests).

Since inhibition was not required for the observed reduction in L29 responses with repeated current injection, we next examined whether intrinsic regulation of L29 excitability could account for these results. These experiments were performed in two parts: in part 1, we repeatedly activated L29s with 30 siphon taps at either a 1 or 30 s ISI; this provides data for comparison to responses obtained with current injection alone. In part 2, we substituted current injection for taps, adjusting the initial current to match the first response observed with siphon tap. Results from these experiments are summarized in **Figure 3A**. At a 1 S ISI, both tap and current resulted in a significant decrease in responding, comparing the 30th response to the first (*n* = 8; tap: *t* = 6.49, current: *t* = 3.81, both *p* < 0.01). However, the decay dynamics between the two training methods differed: first, the two training curves were best fit by different types of equations, with current resulting in a linear decay (*F* = 2.52, *p* = 0.11) and siphon tap producing a

tap-evoked L29 activity is decreased by injection of hyperpolarizing current into both L29 neurons. In the second part of the experiment, tap-evoked L29 activity is decreased through repeated current injection at a 1 s ISI into both neurons (intrinsic plasticity; see **Figure 3**). Repeated current injection also augments the strength of the L29 to LFS excitatory synapse (**Figure 5**). **(B)** Representative physiological traces from a single experiment. All responses are evoked by siphon tap (arrows). LFS MNs are hyperpolarized ≈−40 mV from rest to reveal the siphon-evoked complex EPSP. The dashed line indicates baseline-level responding. The dashed line indicates baseline-level responding. In the HYP TEST, the response to tap in both of the L29s is reduced by hyperpolarizing them. This results in a reduction of the tap-evoked LFS MN response. In phase 2, L29's response

that although the response of L29 to the AUG TEST is decremented to a similar extent as in the HYP TEST, the response of the LFS MN is no longer decremented. **(C)** Summary data of L29 tap-evoked activity from 12 experiments. L29 activity was significantly reduced in both the HYP and AUG tests (\*indicates *p* < 0.01 from baseline using Bonferroni post tests). There was no difference between the HYP and AUG tests, indicating that L29 activity was reduced to a similar extent in both conditions. **(D)** Measures of the tap-evoked complex EPSP in LFS MNs, normalized to their respective baseline measures so that 0% equals no change. Only measures in the HYP test were significantly different than baseline (statistics performed on raw data; \**p* < 0.01 from baseline using Bonferroni post tests).

non-linear, single exponential decay (*F* = 32.77, *p* < 0.01). Second, the decrement observed at the 30th stimulus with siphon tap was significantly greater than that from current injection (*t* = 2.62, *p* < 0.05). Using a 30 s ISI, only training with siphon tap resulted in a significant decrease in responding (*n* = 6; tap: *t* = 5.39, *p* < 0.01; current: *t* = 0.29, *p* = 0.78). The decay during training with tap was best fit with a single exponential decay (*F* = 10.65, *p* < 0.01). Taken together, these data suggest that L29 activity alone can regulate L29 tap-evoked responses, but only ata1s ISI. The differences observed with siphon tap and current injection at a

1 s ISI likely reflect the added contribution of decreased activity in the ULT sensory neurons resulting from training (Fischer et al., 2011).

The regulation observed with current injection at a 1 s ISI could include a contribution from L30-mediated recurrent inhibition, which has been shown to regulate L29 tap-evoked responses following intracellular activation of L29s (Fischer and Carew, 1993). To examine this, we repeated our experiments using current injection in the presence of 100 μm curare (**Figure 3B**). To compare the two training curves directly, we analyzed the data using a two-way repeated measure ANOVA. For the 1 s ISI, we obtained a significant effect of training (*F* = 15.06, *p* < 0.01), but no effect of curare treatment (*F* = 3.22, *p* = 0.10). In contrast, with a 30 s ISI we observed no effect of either training (*F* = 0.92, *p* = 0.99) or curare treatment (*F* =0.00, *p* =0.98). These data suggest that inhibition is not required for the regulation of L29 responses during STH. These data also reinforce our results in illustrated **Figure 2** that inhibition is not required for generalization of training, and suggest that intrinsic regulation of L29 activity alone can account for these results.

In previous work, we demonstrated a role for L30-mediated inhibition in extending the time course of regulation following exposure to water turbulence. Briefly, the time course of recovery of siphon-evoked LFS MN responses was longer than that of siphon-evoked ULT responses following exposure to water turbulence (Calin-Jageman and Fischer, 2007). If L30 neurons were hyperpolarized to prevent their activation by turbulence, the time course of recovery of the MNs shortened to match that of the ULTs (Calin-Jageman and Fischer, 2003a). To see if a similar recovery dynamic is present with STH, we measured the time course of recovery of both the LFS MNs and ULT SNs following training at a 1 s ISI. Each post-training measure (15, 30, 45, and 60 s) was obtained in a separate experiment; subsequent recovery measures were obtained 5 min later. Further, experiments for the two cell types were performed separately; siphon-evoked ULT responses were measured using extracellular recordings from the siphon nerve (as in Calin-Jageman and Fischer, 2003a; Fischer et al., 2011), whereas LFS responses were measured using intracellular recordings.

Summary data from these experiments are presented in **Figure 4**, which depicts the difference from baseline for the post-training test measures (recovery measures are not shown). Statistical analyses were performed on raw data using a one-way repeated measures ANOVA. In all experiments measuring posttraining recovery of LFS responses (**Figure 4**, right), there was a significant overall effect of training (15 s: *n* = 6, *F* = 5.87; 30 s: *n* = 6, *F* = 9.07; 45 s: *n* = 5, *F* = 31.48; 60 s: *n* = 5, *F* = 9.10; all *p* < 0.05). Further, all test responses were significantly reduced from their baseline (pre-training) measures as determined using a Bonferroni post-test (15 s: *t* = 3.31; 30 s: *t* = 4.26; 45 s: *t* = 7.44; 60 s: *t* = 4.08; all *p* < 0.05). Recovery measures were not different from baseline. Conversely, for ULT responses (**Figure 4**, left) only the 15 and 30 s post-training measures exhibited a significant overall effect of training (15 s: *n* = 6, *F* = 8.51, *p* < 0.05; 30 s: *n* = 6, *F* = 6.98, *p* < 0.05; 45 s: *n* = 5, *F* = 3.11, ns; 60 s: *n* = 5, *F* = 0.11, ns). In both the 15 and 30 s recovery experiments, Bonferroni post-tests revealed that test responses were significantly

different from their respective baseline measures (15 s: *t* = 3.99; 30 s: *t* = 3.71; both *p* < 0.05). In all cases, recovery measures were not different from baseline. Thus, there is a difference in the time course of recovery following training between the two cell types, with an extended time of recovery for the LFS MNs. Since the decrease in LFS MN responses cannot be accounted for by the level of ULT activity, inhibition may play a role in extending the time course of regulation, as we had observed in previous experiments using water turbulence (Calin-Jageman and Fischer, 2003a; Calin-Jageman and Fischer, 2007).

#### **SHORT-TERM SYNAPTIC PLASTICITY AND RESPONSE NORMALIZATION**

These experiments examine how site-specificity of training is maintained in the LFS MNs at a 1 s ISI when it is not maintained in the L29s (**Figure 2**; Fischer et al., 2011). Our hypothesis is that augmentation (AUG), a form of short-term synaptic plasticity expressed by the L29s, may act to maintain a constant level of synaptic input to MNs as L29 activity decreases. L29 neurons have been demonstrated to express AUG following a single activation episode with either tail shock or current injection (Frost et al., 1988; Bristol et al., 2001). As a first step, we examined whether habituation training at a 1 s ISI would also result in AUG.

The basic protocol of these experiments is shown in **Figure 5A**. Each L29 – LFS pair was examined both with activation by siphon taps and by current injection (*n* = 9). The L29 – LFS synapse was examined by eliciting a single action potential in the L29s; the "Test" measure following training was obtained 1 s following training to maintain the training interval. Both activation protocols resulted in significant AUG, as illustrated in **Figures 5B** for siphon tap and 5C for current injection (both *p* < 0.01). Post-tests revealed that Test measures were significantly different from Baseline measures in both conditions (tap: *t* = 4.23; current: *t* = 3.50; both *p* < 0.01). Recovery was evident 5 min later, as these measures were not different from Baseline (tap: *t* = 2.09; current: *t* = 1.36). These data demonstrate that the L29 to LFS synapse expresses significant AUG with STH. We did not examine whether AUG was also induced with a 30 s ISI.

We next examined the interaction between AUG and the intrinsic regulation of L29 activity as observed in the untrained pathway with STH. These experiments explored the hypothesis that the induction of AUG could compensate for the decrease in L29 activity to maintain net synaptic input to LFS neurons at a consistent level. Experiments (*n* = 12) were performed in two parts, as illustrated in **Figure 6A**: in part one (HYP Test), we examined whether a decrease in L29 tap-evoked responses alone would result in a significant change in the LFS MN complex EPSP. There are five known L29 interneurons within the abdominal ganglion (Hawkins et al., 1981; Frost and Kandel, 1995). While hyperpolarizing a single L29 can have a significant effect on the MN tap-evoked complex EPSP (Fischer and Carew, 1993), this effect required complete inactivation of the L29s, and our goal here was to only reduce activity, not eliminate it. We therefore chose to simultaneously hyperpolarize two L29s in these experiments. We also ensured that both L29s provided synaptic input to the MNs (mean: 5.1 ± 2.5 mV; range = 1.3–9.4 mV). Recovery measures obtained 5 min later were performed with both

L29s again at rest; this measure also served as the baseline for the second part of the experiment. In part 2 (AUG Test), we activated both of the L29s with current injection at a 1 s ISI to produce a decrease in L29 activity equivalent to that obtained in part 1. This activation would also induce AUG in the L29s, as was illustrated in **Figure 5**. The question is whether this addition of AUG would compensate for the decrease in L29 activity to maintain the tap-evoked complex EPSP in LFS MNs at baseline levels.

We chose to hyperpolarize both of the L29s to reduce activity in each by approximately two spikes, the average reduction observed at the end of training with current at a 1 s ISI (**Figure 3A**). Summary data illustrating the decrease in L29 activity in these experiments are illustrated in **Figure 6C**. On average, hyperpolarization decreased tap-evoked activity in the L29s by 2.1 ± 1.3 spikes, and training in the AUG test decreased activity by 2.0 ± 1.6 spikes. We analyzed tap-evoked L29 activity across the two parts of this experiment with a one-way ANOVA. We observed an overall significant effect of test condition (*F* = 21.66, *p* < 0.01). Post-tests revealed that L29 activity was significantly reduced compared to baseline in both the HYP and AUG tests (HYP: *t* = 6.35; AUG: *t* = 5.97; *p* < 0.01 for both). Importantly, there was no difference between the level of reduction comparing the HYP and AUG tests (*t* = 0.38, ns). Recovery measures were not different from baseline.

As shown in **Figure 6B**, reducing L29 activity by just two spikes in the HYP test results in a decrease in the tap-evoked complex EPSP in siphon MNs. Quantitative data from our 12 experiments are presented in **Figure 6D**; data are normalized to their respective baseline measures so that 0% represents no change. On average, the MN complex EPSP was reduced by −23.5 ± 4.1% of baseline with L29 hyperpolarization. Conversely, no reduction was observed when L29 tap-evoked activity was reduced by a similar extent following training with current injection (AUG test); on average, the MN complex EPSP was 0.9 ± 4.8% of baseline. Complex EPSP data from these experiments were analyzed with a one-way ANOVA (this analysis was performed on the raw data, not the data normalized to baseline). We observed an overall significant effect of test condition (*F* = 10.21, *p* < 0.01). Post-tests revealed a significant difference between the HYP test and Baseline (*t* = 4.57, *p* < 0.01), HYP test and AUG test (*t* = 4.24, *p* < 0.01) and HYP test and Recovery (*t* = 4.69, *p* < 0.01). No other comparisons reached significance. Since the major difference between the two tests is the induction of AUG, these data demonstrate that this form of short-term synaptic plasticity can compensate for the reduced activity of the L29 excitatory interneurons, effectively normalizing net synaptic input to maintain a constant level.

#### **DISCUSSION**

We have examined STH of the siphon-elicited siphon withdrawal (S-SWR) as a means to characterize dynamic changes within a neural network as it adjusts to accumulating sensory input. Our interest in STH was driven by previous research examining the impact of water turbulence on regulating siphon withdrawal ("environmental regulation"), which produces a continuous and complex form of low-threshold sensory stimulation (Fischer et al., 2000). The comparison was of interest to us because both result in a common behavioral outcome, and both have the net result of optimizing behavioral responding based on the recent history of sensory input (Fischer et al., 2000; Calin-Jageman and Fischer, 2003b). STH provides a means to assess the functional interaction between multiple network processes that include sensory regulation, intrinsic forms of plasticity, and synaptic inhibition under conditions where the temporal patterning of stimuli can be more tightly controlled, and the spatial extent of stimulation is more restricted.

Our focus here was on the L29 excitatory interneurons, which are a major source of excitatory input to siphon MNs (Fischer and Carew, 1993; Frost and Kandel, 1995). Our results show that two activity-dependent processes intrinsic to the L29s, excitability (or intrinsic plasticity: Zhang and Linden, 2003) and a form of shortterm synaptic plasticity (AUG) both contribute to SWR network dynamics during STH at rapid (1 s ISI) training intervals, but not at longer (30 s ISI) intervals. Intrinsic plasticity is a commonly observed property in neurons that operates in parallel with synaptic changes to regulate neural networks in an activity-dependent fashion (for reviews see Zhang and Linden, 2003; Benjamin et al., 2008; Sehgal et al., 2013). The L29s have previously been shown to exhibit significant spike frequency adaptation to a single (5 s) current pulse with L30-mediated inhibition blocked by curare (Lieb and Frost, 1997). Our results demonstrate a similar regulation of excitability in the absence of synaptic inhibition to repeated activation during training. This decrease in excitability results in a decreased L29 response to siphon tap, which *if expressed alone* would result in decrease in excitatory input to siphon MNs (as demonstrated in **Figure 5C**).

Our results also demonstrated significant AUG of the L29- LFS MN excitatory synapse at fast (1 s) but not slower (30 s ISI) training intervals. This co-expression of AUG with intrinsic plasticity acts to maintain a stable level of synaptic input to the MNs in the untrained pathway, compensating for the decreased tap-evoked activity of the L29s. A similar interaction has been described in studies of neurons in the crab stomatogastic ganglion utilizing the dynamic clamp technique, where changes in synaptic conductance were shown to compensate for significant variations in intrinsic excitability (Grashow et al., 2010). In a similar manner, we have previously characterized an interaction between AUG and extrinsic regulation of the L29s via the neuromodulator serotonin (5-HT). Bath application of 5-HT results in a decrease in L29 excitability and produces a parallel decrease in the L29 to LFS MN monosynaptic EPSP. Tail shock (which releases 5-HT: Marinesco et al., 2004) produced similar effects, but only in L29 neurons that exhibited little to no action potential activity during the administration of shock. In L29 neurons that responded vigorously to tail shock, significant synaptic enhancement (AUG) was observed that offset the modulatory effects of 5-HT on L29 synaptic transmission (Bristol et al., 2001). Taken collectively, these results reinforce the concept that understanding the contribution of any one form of plasticity at any given neural locus can only be made within the context of other forms of plasticity concurrently active both within single neurons and throughout a neural network (Calin-Jageman and Fischer, 2003a; Marder, 2011).

#### **MECHANISMS OF SHORT-TERM HABITUATION**

The gill and siphon withdrawal reflexes in *Aplysia* provided one of the first model systems to explore synaptic mechanisms underlying habituation. While a primary emphasis in the literature has been on the monosynaptic connection from LE sensory neurons to MNs (for review see Glanzman, 2009), the overall roster of contributing mechanisms will depend on the intensity of the training stimulus, the training interval used, and the site of training on the body surface. An important line of evidence that implicates regulated sensory processing, as a primary overall mechanism is that habituation is training site-specific when both sites are on the siphon itself (Frost et al., 1997; Ezzeddine and Glanzman, 2003; Fischer et al., 2011). Conversely, STH of tail stimulus-elicited siphon withdrawal (T-SWR) appears to be mediated through interneurons (Stopfer and Carew, 1996), and has a requirement for inhibition in the abdominal ganglion that is not shared with STH of siphon-elicited siphon withdrawal (Bristol and Carew, 2005). These observations demonstrate that different network elements can contribute depending upon the particular sensory pathway used for training.

As illustrated in **Figure 1B**, the LE and ULT sensory neurons form parallel pathways from the siphon that differ in their stimulus thresholds (Frost et al., 1997; Hickie et al., 1997; Illich and Walters, 1997; Walters and Cohen, 1997; Calin-Jageman and Fischer, 2007). The relative contribution of each to habituation will depend on the extent a stimulus activates the LE neurons, assuming that the lower-threshold ULTs will be activated by all stimuli capable of activating the LEs (Fischer et al., 2011). This dependence on intensity complicates the ability to isolate the net contribution of the LEs in the absence of ULT activity. It is also difficult to estimate the net ULT contribution to siphon-evoked responses, since monosynaptic EPSPs from these cells have yet to be measured, and the number of cells activated by a stimulus is not known. Further, up to 80% of the siphon-evoked complex EPSP may be mediated through interneurons (Trudeau and Castellucci, 1992). Of the remaining 20%, LEs are estimated to contribute around 5% of the complex EPSP (Hickie et al., 1997; Walters and Cohen, 1997), suggesting that the direct ULT to MN connection also plays an important regulatory role.

The cellular form of plasticity that contributes to habituation appears to differ between the two sensory neuron types. The regulation of the monosynaptic EPSP between LE neurons and MNs with habituation has been well documented (Castellucci et al., 1970, 1978; Castellucci and Kandel, 1974; Byrne et al., 1978b; Byrne, 1982; Cohen et al., 1997; Ezzeddine and Glanzman, 2003). While the LE to MN synaptic efficacy decreases with training, evoked action potential activity of the LEs during training appears to be stable (Byrne et al., 1978a). In contrast, our previous data demonstrated that ULT regulation is based primarily on adjusting the level of activity to match the salient stimulus characteristics of the environment, be it the presence of water turbulence or a particular STH training interval. This change in the level of activity alone could account for training-induced regulation of the SWR network with a 30 s ISI, but not at a 1 s ISI (Fischer et al., 2011) which invokes intrinsic plasticity expressed by the L29s into the regulatory mix, as illustrated in the present work.

The differing cellular mechanisms expressed by the ULTs and LEs raises an interesting possibility on the contribution of these processes to short-and long-term regulation (e.g., within and between sessions habituation). Synaptic depression in LE neurons is restricted to the site of training, and can effectively serve as a long-term "mark" of this experience (Frost et al., 1997; Ezzeddine and Glanzman, 2003). The regulation of ULT activity is well suited to mediate short-term changes, since the level of ULT activity can dynamically adjust to directly reflect the temporal dynamics of tactile stimulation (Calin-Jageman and Fischer, 2007; Fischer et al., 2011). Assuming that the ULTs do not exhibit long-term changes in activity, these mechanisms combined would allow the network to continue to exhibit short-term regulation via regulated ULT activity even when the synaptic efficacy of LE neurons is depressed to reflect the "memory" of the site of habituation training.

# **ACKNOWLEDGMENT**

Supported by N.S.F. grant #IBN-0110372 to Thomas M. Fischer.

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 06 September 2013; accepted: 28 January 2014; published online: 13 February 2014.*

*Citation: Fischer TM, Jacobson DA and Demorest-Hayes K (2014) Network processes involved in the mediation of short-term habituation in Aplysia: contribution of intrinsic regulation of excitability and synaptic augmentation. Front. Integr. Neurosci. 8:15. doi: 10.3389/fnint.2014.00015*

*This article was submitted to the journal Frontiers in Integrative Neuroscience.*

*Copyright © 2014 Fischer, Jacobson and Demorest-Hayes. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Habituation of reflexive and motivated behavior in mice with deficient BK channel function

# *Marei Typlt 1, Magdalena Mirkowski 1, Erin Azzopardi 1, Peter Ruth2 , Peter K. D. Pilz <sup>3</sup> and Susanne Schmid1\**

*<sup>1</sup> Department of Anatomy and Cell Biology, Schulich School of Medicine and Dentistry, University of Western Ontario, London, ON, Canada*

*<sup>2</sup> Pharmakologie and Toxikologie, Pharmazeutisches Institut, Universität Tübingen, Tübingen, German*

*<sup>3</sup> Tierphysiologie, Zoologisches Institut, Universität Tübingen, Tübingen, German*

#### *Edited by:*

*Donald A. Wilson, New York University School of Medicine, USA*

#### *Reviewed by:*

*Dennis Eckmeier, Cold Spring Harbor Laboratory, USA Benjamin Sadrian, Nathan Kline Research Institute, USA*

#### *\*Correspondence:*

*Susanne Schmid, Department of Anatomy and Cell Biology, Schulich School of Medicine and Dentistry, University of Western Ontario, Medical Sciences Building, Room 470, London, ON N6A 5C1, Canada e-mail: susanne.schmid@ schulich.uwo.ca*

Habituation is considered the most basic form of learning. It describes the decrease of a behavioral response to a repeated non-threatening sensory stimulus and therefore provides an important sensory filtering mechanism. While some neuronal pathways mediating habituation are well described, underlying cellular/molecular mechanisms are not yet fully understood. In general, there is an agreement that short-term and long-term habituation are based on different mechanisms. Historically, a distinction has also been made between habituation of motivated versus reflexive behavior. In recent studies in invertebrates the large conductance voltage- and calcium-activated potassium (BK) channel has been implicated to be a key player in habituation by regulating synaptic transmission. Here, we tested mice deficient for the pore forming α-subunit of the BK channel for shortterm and long-term habituation of the acoustic startle reflex (reflexive behavior) and of the exploratory locomotor behavior in the open field box (motivated behavior). Short-term habituation of startle was completely abolished in the BK knock-out mice, whereas neither long-term habituation of startle nor habituation of motivated behavior was affected by the BK deficiency. Our results support a highly preserved mechanism for short-term habituation of startle across species that is distinct from long-term habituation mechanisms. It also supports the notion that there are different mechanisms underlying habituation of motivated behavior versus reflexive behavior.

**Keywords: BK channel, sensorimotor gating, habituation, locomotion, startle**

# **INTRODUCTION**

The brain constantly receives a vast amount of sensory information. In order to be able to extract salient information and respond appropriately, it is necessary to suppress repetitive noninformative input. One important sensory filtering mechanism responsible for suppression is habituation. Habituation describes the progressive decrease of a behavioral response to repetitive non-threatening sensory stimuli. It is considered to be the most basic form of learning and allows to ignore irrelevant stimuli in favor of relevant stimuli (Poon and Young, 2006). It is further believed to be a prerequisite for other learning forms (Rankin et al., 2009). In humans, disruption of habituation is strongly correlated with cognitive impairments. This was found in patients with mental disorders like schizophrenia (Geyer and Braff, 1982; Ludewig et al., 2003; Takahashi et al., 2008) and autism spectrum disorders (Ornitz et al., 1993; Perry et al., 2007).

While habituation is well characterized behaviorally (Thompson and Spencer, 1966; Groves and Thompson, 1970; Davis, 1984; Rankin et al., 2009), the underlying cellular/molecular mechanisms are not yet fully understood. Williams et al. (1974, 1975) suggested two different mechanisms of habituation by showing that habituation of motivated behavior was dependent on cholinergic mechanisms while the habituation of reflexive behavior was not. Also, the mechanisms for habituation within a session (shortterm habituation) and between sessions (long-term habituation) seem to differ since they occur at different time scales and are largely independent from each other (Davis, 1984; Leaton et al., 1985).

Whereas mechanisms underlying habituation of motivated behavior have rarely been studied, habituation of reflexive behavior, especially of the startle response or similar escape responses, is relatively well studied in invertebrates and vertebrates, including humans. A calcium-dependent presynaptic depression mechanism in sensorimotor synaptic terminals within the primary startle pathway has been proposed to mediate short-term habituation of startle (Weber et al., 2002; Simons-Weidenmaier et al., 2006; Gover and Abrams, 2009). However, previous data have also shown that a common cause of synaptic depression, namely vesicle depletion, is unlikely to contribute substantially to the synaptic depression underlying habituation (Castellucci and Kandel, 1974; Byrne, 1982; Weber et al., 2002); the cause for synaptic depression therefore remains elusive. Recently, it has been shown that a loss-of-function mutation of the large conductance voltage- and calcium-activated potassium (BK) channel impairs short-term habituation of an escape response in *Drosophila* (Engel and Wu, 1998). BK channels are expressed throughout the mammalian nervous system (Knaus et al., 1996; Wanner et al., 1999; Sausbier et al.,

"fnint-07-00079" — 2013/11/15 — 21:59 — page 1 — #1

2006) and can be found at neuronal soma and processes, as well as on presynaptic terminals (Knaus et al., 1996; Misonou et al., 2006; Sailer et al., 2006). They can be activated by membrane depolarization and micromolar concentrations of intracellular calcium and therefore can establish a link between intracellular free calcium, electrical excitability, and transmitter release in synaptic terminals (Robitaille et al., 1993; Yazejian et al., 1997; Sailer et al., 2006).

We here investigate the role of BK channels in both habituation of motivated and of reflexive behavior in mammals using mice that lack functional BK channels. We measured motor activity in a locomotor box, which reflects exploratory behavior in rodents (Crawley and paylor, 1997), in order to quantify motivated behavior. For measuring reflexive behavior, we used the acoustic startle response. We hypothesized that BK channel deficient mice show a disruption of short-term habituation of the startle response, corresponding to findings in *Drosophila* and *Caenorhabditis elegans*. We then tested whether BK channel knock-out mice also show disruptions in long-term habituation of startle. Finally, we analyzed whether BK channels play a role in habituation of motivated behavior, measuring exploratory behavior of BK channel knock-out mice in the locomotor box.

# **MATERIALS AND METHODS**

#### **ANIMALS AND ANIMAL CARE**

We used mice of the F1 generation of a hybrid SV129/C57BL6 line with a deficient BK channel function bred at University of Tübingen. The BK channel function was abolished by deleting the *slo1* gene (accession ID# AAD49225.1) which encodes the pore forming channel protein (α-subunit; for details about generation of mice and genotyping please see supporting information in Sausbier et al., 2004). Heterozygous C57Bl6 mice were paired with heterozygous SV129 mice. Exclusively mice of the respective F1 generation, wild-type (WT), heterozygous (BKα+/−), and homozygous knock-out (BKα−/−), were tested in order to avoid effects of inbreeding. The animals were litter- and/or age matched. We only tested mice at ages from 3 to 5 months to avoid effects of aging.

All mice were generated and genotyped at the Pharmaceutical Institute, University of Tübingen, Germany. They were ear-tagged and shipped to Canada at the age of 1.5–3 months and subsequently quarantined and allowed to acclimate for 2 weeks before behavioral testing started. Mice were group housed with mixed genetic background within groups, with a 12 h light–dark cycle and with *ad libitum* food and water. Tails were marked according to their ear-tags for easy identification. Behavioral testing occurred during the light cycle. After all behavioral testing was finished, mice were sacrificed, and genotype was once more verified, comparing the ear-tags, shipping list, and tail marks.

All procedures were in accordance with the ethical guidelines of the Canadian Council on Animal Care (CCAC) and approved by the University of Western Ontario Animal Use Subcommittee.

#### **ACOUSTIC STARTLE REFLEX**

Reflexive behavior was measured using the acoustic startle reflex. 18 WT (10 males/8 females), 17 BKα+/<sup>−</sup> (10/7), and 19 BKα−/<sup>−</sup> mice (9/10) were tested as described previously (Geyer and Swerdlow, 2001; Valsamis and Schmid, 2011). Testing was conducted in sound attenuated startle boxes from MED Associates (MED-ASR-PRO1, St Albans, VT, USA), where animals were placed into small encasements mounted on a movement sensitive platform within a sound attenuated chamber. A piezoelectric transducer mounted below the platform converted vertical movements of the platform induced by startle responses of the mouse into a voltage signal. The maximum amplitude (positive peak to negative peak) of the signal was measured in a 100 ms time window after the acoustic stimulus onset, using the associated software for stimulus presentation and recordings (see **Figure 1**; Startle Reflex version 6.0, MED Associates, Inc.).

Before the actual testing the animals were acclimatized to the startle boxes for 5 min on three consecutive days. During the acclimation periods only the background noise (65 dB sound pressure level, SPL, white noise) was presented. On the third day, acclimation was followed by a short input/output (I/O) test to determine the appropriate gain setting for amplifying the voltage signal of the transducer for each individual animal, so that a large portion of the dynamic range of the startle system was used for measuring startle in each animal: the I/O test consisted of 12 startle stimuli with increasing intensity starting at 65 dB SPL and increasing by 5 dB SPL each trial to 120 dB SPL (20 ms duration, white noise, every 20 s), presented on top of the background noise. The gain was set so that the maximum startle amplitude would cover a large portion of the dynamic range and the gain was subsequently kept constant for all recordings of a given animal. The absolute startle response amplitude was later corrected for the gain factor.

On the next five consecutive days the animals were tested using the following protocol: the animals were acclimatized to the startle box and the background noise for 5 min. Subsequently, the startle stimulus (20 ms, 105 dB SPL white noise) was presented 100 times with varying inter-trial intervals (10–20 s) on top of the background noise. Recordings started 50 ms before the stimulus was given and lasted for a total of 500 ms. To account for the muscular tremor occurring in the BKα−/<sup>−</sup> mice, we subtracted the peak-topeak transducer displacement during the phase before the startle

**FIGURE 1 | Acoustic startle response measurement.** One acoustic startle response, measured by the Med Associates startle box, of a BK knock-out mouse (BKα−/−, **A**), and of a wild-type animal (WT, **B**), both recorded with a transducer signal gain of 1. The red line shows the voltage signal produced by the transducer that reflects the vertical dislocation of the platform by the animals' movement, and the short horizontal black bar represents the startle stimulus. The blue arrow indicates the peak-to-peak amplitude measured by the system. The black diamonds indicate the baseline noise generated by the knock-out mouse's tremor that was subtracted from the total startle amplitude.

"fnint-07-00079" — 2013/11/15 — 21:59 — page 2 — #2

pulse (50 ms) from the displacement measured during the startle pulse (see **Figure 1A**).

#### **OPEN FIELD LOCOMOTOR ACTIVITY**

Motivated behavior was measured using open field locomotor activity that reflects exploratory behavior. Locomotor activity was measured in 16 mice of each genotype (WT: nine males/seven females, BKα+/−: 10/6, BKα−/−: 9/7). Each animal was placed in a squared (40 cm × 40 cm) open field box (Versamax animal activity monitor, AccuScan Instruments, Columbus, OH, USA) in a dimly lit room for five consecutive days and was allowed to explore freely for 2 h. In order to assess habituation, the distance traveled during these 2 h was analyzed in 5 min blocks using the VersaMaxTM software (AccuScan Instruments).

#### **DATA ANALYSIS AND STATISTICS**

Data analyses and graphical display were done with Microsoft Excel (version 14.0.6129.5000, Microsoft Corp.), GraphPad (for graphic display only, Prism 6.01, GraphPad Software, La Jolla, CA, USA) and SPSS (for statistical analysis, version 20.0.0, IBM Corp.). Data for habituation curves are expressed as mean ± SE. For comparisons of baseline startle and habituation scores (see below), data is displayed as median and 25%/75% quartiles in box–whisker plots with whiskers indicating the 95th and 5th percentile.

In order to account for differences in the baseline, data of five subsequent trials were always averaged to a block value and these values were normalized to the first block within each day for assessing short-term habituation. For assessing long-term habituation, block one of each day was normalized to the first block on day 1. In order to compare the values across trials or days a repeated measurement ANOVA with genotype and gender as between-subjects factors was performed. We did not find a gender effect for any of the tested parameters, thus they were not reported separately in the Section "Results." The Mauchly test was used to judge if the data violated the sphericity assumption. In case of a violation the degrees of freedom were corrected using the Greenhous–Geisser (if < 0.75) or the Huynh–Feldt method (if > 0.75).

In order to evaluate the amount of habituation we calculated a score for both, short- and long-term habituation. For short-term habituation the score is calculated as the ratio between the last block and the first block, whereasfor long-term habituation it is the ratio between the first block of the last day of testing (day 5) and the first block of the first day of testing. A score of 1 indicates no habituation. The smaller the score, the more the animal habituated. A score > 1 indicates sensitization. The scores between genotypes were compared using a two-way-ANOVA (genotype × gender).

For *post hoc* analyses the Sidak *t*-test was performed to account for repetitive testing. Differences were considered statistically significant when *p*-values were smaller than 0.05. In the figures significance levels between genotypes were indicated as followed: \**p* ≤ 0.05, \*\**p* ≤ 0.01, \*\*\**p* ≤ 0.001.

# **RESULTS**

We tested the F1 generation of hybrid SV129/C57BL6 homozygous BKα knock-out mice (BKα−/−), as well as their heterozygous

(BKα+/−), and WT littermates for short-term and long-term habituation of reflexive and motivated behavior.

#### **HABITUATION OF REFLEXIVE BEHAVIOR**

\*\*\**p* < 0.001.

The startle reflex to a sudden acoustic stimulus significantly decreased in the WT and BKα+/<sup>−</sup> mice across testing trials within a session, but not in the BKα−/<sup>−</sup> mice (**Figure 2**). A repeated measurement ANOVA (genotype × gender × trials) on amplitudes normalized to the first five trials (**Figure 3A**) reported a main effect for trial (*F*(11.44,571.76) = 5.58, *p* < 0.001), as well as a significant difference between genotypes (*F*(2,50) = 8.57, *p* = 0.001) and an interaction between both (*F*(22.87,571.75) = 1.74, *p* = 0.018). A *post hoc* test confirmed that habituation of the startle amplitudes in the BKα−/<sup>−</sup> mice were significantly different from that of their WT littermates (*p* < 0.001).

In order to quantify the amount of habituation occurring, we calculated the short-term habituation scores as the ratio between the average of the last five and the first five trials. The ANOVA on these scores showed a significant effect for genotype (*F*(2,50) = 5.10, *p* = 0.010). WT mice showed an average score of 0.70 ± 0.08 SE, which means they habituated on average by 30%. In contrast, the BKα−/<sup>−</sup> mice did not habituate at all, with an average score of 1.03 ± 0.07 SE (*p* = 0.009). The scores of the heterozygote BKα+/<sup>−</sup> mice fell in between the scores of WT and BKα−/<sup>−</sup> mice and were not significantly different from either (*p* = 0.757, *p* = 0.119, respectively). **Figure 3B** shows the median habituation scores in a box–whisker plot in order to give an idea about the distribution of scores across animals.

Notably, the absolute startle amplitudes were significantly lower in the BKα−/<sup>−</sup> mice (average 148 <sup>±</sup> 9 mV SE) compared to their WT litter mates (273 ± 30 mV SE). A two-way ANOVA on the absolute startle amplitudes resulted in a significant genotype effect (*F*(2,50) = 6.57, *p* = 0.003) and *post hoc* analysis showed that the difference was between the WT and the BKα−/<sup>−</sup> mice (*<sup>p</sup>* <sup>=</sup> 0.001,

"fnint-07-00079" — 2013/11/15 — 21:59 — page 3 — #3

**FIGURE 3 | Short-term habituation of the acoustic startle reflex. (A)** Of the 100 normalized startle response amplitudes of BKα−/−, BKα+/−, and WT mice, five consecutive startle responses were always averaged to one block. WT mice show a decline in response amplitude, whereas knock-out animals show no decline, with heterozygous animals intermediate. Error bars indicate standard error. \*\*\**p* = 0.001. In **(B)** the habituation scores of the respective genotypes are displayed (first five versus last five responses,

see Materials and Methods" for detailed description). Data is displayed as median in a box–whisker plot, with whiskers indicating the 5th and 95th percentile. \**p* = 0.009. **(C)** The baseline startle response was significant lower in knock-out animals than in WT. Data is displayed as median with whiskers indicating the 5th and 95th percentile. 18 WT (10 males/8 females), 17 BKα+/<sup>−</sup> (10/7), and 19 BKα−/<sup>−</sup> mice (9/10) were tested. \*\*\**p* = 0.001.

see **Figure 3C**). Still, the startle amplitudes of the BKα−/<sup>−</sup> mice were well above the noise level, which was at 43 ± 1 mV SE.

Across 5 days of testing the initial startle amplitude increased in all animals suggesting that they sensitized to the stimulus between sessions rather than habituated. In order to account for the difference in baseline startle, amplitudes of the first block of each day were normalized to the first block of day 1 for each genotype. A repeated measurement ANOVA (genotype × gender × day) on normalized data reported an effect of day (*F*(4,160) = 9.56, *p* < 0.001), but no significant effect of the genotype (*F*(2,40) = 2.43, *p* = 0.101), nor a genotype × day interaction (*F*(8,160) = 1.236, *p* = 0.281, **Figure 4A**). Accordingly, also the long-term habituation scores were not significantly different between genotypes (*F*(2,40) = 0.785, *p* = 0.463, **Figure 4B**).

#### **HABITUATION OF MOTIVATED BEHAVIOR**

Locomotor activity as a measure for exploratory behavior was assessed in a locomotor box (Crawley and paylor, 1997). Within the 2 h test in the open field box all animals habituated to the environment, leading to a strong decline in locomotion (**Figure 5A**). A repeated measurement ANOVA (genotype × gender × time) on the distance traveled within 5 min normalized to the distance traveled in the first 5 min showed a significant effect of time (*F*(9.2,387.7) = 31.45, *p* < 0.001), but no genotype effect (*F*(2,42) = 1.50, *p* = 0.067), or a genotype × time interaction (*F*(18.5,387.7) = 0.65, *p* = 0.860). In consequence, there was no significant difference between genotypes in the short-term habituation scores for the locomotor behavior (*F*(2,42) = 0.848, *p* = 0.435), all genotypes had a similar score of around 0.2 (**Figure 5B**).

The absolute amount of locomotion was statistically not significantly different between genotypes. A two-way ANOVA on the absolute locomotion resulted in no significant genotype effect (*F*(2,42) = 1.890, *p* = 0.164, **Figure 5C**).

Also the habituation across days did not significantly differ between genotypes for the locomotor behavior. The respective repeated measurement ANOVA (genotype × gender × day) only showed an effect for the day (*F*(3.4,139.6) = 7.93, *p* < 0.001), but not for genotype (*F*(2,41) = 2.72, *p* = 0.078). There also was no genotype × day interaction (*F*(6.8,139.6) = 1.067, *p* = 0.39, **Figure 6A**). The long-term habituation scores were also statistically not significantly different between the genotypes (*F*(2,41) = 2.41, *p* = 0.102, **Figure 6B**).

19 BKα−/<sup>−</sup> mice (9/10) were tested.

"fnint-07-00079" — 2013/11/15 — 21:59 — page 4 — #4

**FIGURE 6 | Long-term habituation of open field locomotor activity**. **(A)** Changes in average locomotion across days showed no statistically significant difference between genotypes. Error bars indicate standard errors. **(B)** The long-term habituation scores (median, with whiskers indicating the 5th and 95th percentile) were also not significantly different between the genotypes. Knock-out animals show a slightly higher score (less long-term habituation), which is mainly due to an initial sensitization on day 2, followed by a decline.

In summary, our study reveals and impact of deficient BK channel function on short-term habituation of the acoustic startle response, but not on short-term habituation of exploratory behavior, nor on long-term habituation.

#### **DISCUSSION**

The present study shows a lack of short-term habituation of startle responses in BK channel knock-out mice, indicating a crucial role for BK channels in short-term habituation of reflexive responses, but no alterations in long-term habituation of startle or in habituation of exploratory behavior.

#### **LACK OF SHORT-TERM HABITUATION OF STARTLE IN BKα−/<sup>−</sup> MICE**

Our data clearly show that the BKα−/<sup>−</sup> mice we used startled in response to a sound stimulus and that no habituation of this startle occurred. Several difficulties had to be considered in experimental design: it has been shown that a BK channel deficiency can alter locomotion (Sausbier et al., 2004) and hearing (Rüttiger et al., 2004; Oliver et al., 2006; Pyott et al., 2007; Kurt et al., 2012) which could affect the acoustic startle measures (as well as exploratory behavior in the locomotor box). We therefore used a F1 hybrid mouse in present study that has no hearing impairment in the relevant frequency spectrum (Typlt et al., 2013). Still, BKα−/<sup>−</sup> mice showed a lower baseline startle response than their WT siblings. It is difficult to determine if this is due to lower body weight in BKα−/<sup>−</sup> mice, lower anxiety levels, or motor impairments. We accounted for the lower baseline startle amplitude by normalizing the data of each mouse to its startle amplitude in the first trials. However, lower startle responses in general may influence the amount of habituation. The major concern is a floor effect, i.e., that startle response amplitude may not be sufficiently different from a general noise level and can therefore not be further reduced. However, the noise level in our experiments was still considerably lower than startle amplitudes of BKα−/<sup>−</sup> mice. Furthermore, we subtracted the noise caused by tremor in knock-out mice so that it does not influence habituation scores. It also has been shown in the same BKα−/<sup>−</sup> mice that startle responses are substantially reduced by prepulse inhibition (Typlt et al., 2013), indicating that there is still enough room for a substantial reduction of startle, which makes it unlikely that a floor effect accounts for the lack of habituation. We also looked at habituation exclusively in WT low startler with a baseline startle response comparable to BKα−/−. Although the number of WT low startler is too small to statistically analyze the data (*n* = 5), they all have habituation scores well below 1 (data not shown). We are therefore confident that there is a true lack of habituation in BKα−/<sup>−</sup> mice.

#### **POSSIBLE ROLE OF THE BK CHANNELS IN SHORT-TERM HABITUATION OF STARTLE**

It has been proposed that a calcium-dependent presynaptic depression mechanism in sensorimotor synaptic terminals within

"fnint-07-00079" — 2013/11/15 — 21:59 — page 5 — #5

the primary startle pathway mediate short-term habituation of startle (Weber et al., 2002; Simons-Weidenmaier et al., 2006; Gover and Abrams, 2009). BK channels that are activated by depolarization and intracellular calcium drive the membrane potential towards the potassium equilibrium potential and therefore re- and hyperpolarize the neuron (Kaczorowski et al., 1996; Vergara et al., 1998; Poolos and Johnston, 1999; Hu et al., 2001). By limiting the duration of action potentials, they regulate the general excitability of neurons (Shao et al., 1999; Nelson et al., 2003; Brenner et al., 2005), as well as limit the transmitter release at presynaptic terminals (Robitaille and Charlton, 1992; Robitaille et al., 1993; Hu et al., 2001; Raffaelli et al., 2004; Wang, 2008). They colocalize with voltage-gated calcium channels at the active synaptic zone, establishing a link between intracellular free calcium and neurotransmitter release in synaptic terminals (Robitaille et al., 1993; Yazejian et al., 1997; Sailer et al., 2006). All these described properties make BK channels excellent candidates for mediating calcium-dependent synaptic depression in the startle pathway thereby causing habituation to repetitive strong stimuli.

Short-term habituation lasts for several minutes, whereas intracellular calcium is elevated in synaptic terminals only for milliseconds. So, how is synaptic depression maintained for minutes? BK channels can be phosphorylated by PKA, PKC, and most importantly by Ca2+/calmodulin dependent protein kinase II (CaMKII; Kaczorowski et al., 1996; Liu et al., 2007; Wang, 2008). The latter has been shown to be enriched in presynaptic terminals (Gorelick et al., 1988; Walaas et al., 1989) and to act as a strong regulator of synaptic strength and plasticity (Wang, 2008). CaMKII is activated by elevations of intracellular calcium and can auto-phosphorylate upon large calcium accumulation. Autophosphorylation leads to a prolonged activity of CaMKII that persists after calcium levels have returned to baseline. The prolonged activity of CaMKII leads to a prolonged phosphorylation of BK channels and therefore to a lasting decrease in synaptic efficacy (Wang, 2008). In fact, this proposed mechanism meets all requirements for a habituation mechanism, such as a presynaptic localization, calcium dependence, and the reversibility of the phosphorylation process within a timescale of several minutes.

Short-term habituation of startle in rodents has been shown to be mediated at the sensorimotor synapse in the pontine reticular formation where synaptic depression occurs upon repeated strong stimulation (Davis et al., 1982; Lingenhöhl and Friauf, 1992, 1994; Weber et al., 2002; Simons-Weidenmaier et al., 2006). Since the phosphorylation of BK channels requires a strong activation as described above, they are likely to mediate synaptic depression at this synapse, however, future electrophysiological experiments have to confirm this. It will also be intriguing to see in the future to what extend habituation of other reflexive behaviors depend on BK channel activation.

#### **POTENTIAL ROLE OF THE BK CHANNELS IN HABITUATION OF MOTIVATED BEHAVIOR**

BK channels can potentially control transmitter release at any kind of synapse regardless of the type of neurotransmitter released. Furthermore, BK channels are expressed throughout the nervous system, so it could be hypothesized that their activation represents a universal mechanism for habituation. In our study, however, we found an effect of a functional BK channel deficiency only for short-term habituation of startle and not for long-term habituation nor habituation of motivated behavior. In fact, habituation of motivated behavior has been previously suggested to be based on separate mechanisms to habituation of reflexive behavior (Williams et al., 1974, 1975; Brown, 1976). Moreover, the proposed BK channel mechanism is unlikely to be able to mediate motivated behavior since in contrast to reflexive behavior there is no strong eliciting input for motivated behavior which could trigger the phosphorylation of CaMKII. Thus, a different mechanism is likely to account for habituation of motivated behavior.

#### **POTENTIAL ROLE OF THE BK CHANNELS IN LONG-TERM HABITUATION**

Long-term habituation of startle has been shown to be located extrinsically to the primary startle pathway and involves the cortex and the cerebellar vermis (Leaton and Supple, 1986, 1991; Lopiano et al., 1990) and potentially cholinergic mechanisms (Schmid et al., 2011). It has been hypothesized to be an associative learning process. Since associative learning is affected by a lack of BK channel function (Matthews and Disterhoft, 2009; Typlt et al., 2013) we would have expected to see an effect of the BK channel deficiency on long-term habituation of startle as well. Unfortunately, neither WT- nor BK-deficient animals really showed long-term habituation of startle, which is common for many mouse strains. This makes it difficult to assess any differences between genotypes. The lack of statistically significant differences between genotypes in long-term habituation testing of both startle and locomotor behavior does therefore not completely rule out that there is a potential contribution of BK channels, for instance through their impact on associative learning

# **CONCLUSION**

The results of this study show that BK channel activation is necessary for short-term habituation of startle. It demonstrates that this mechanism underlying short-term habituation is highly preserved throughout evolution, since BK channel-dependent short-term habituation of a startle-like response has been found in *C. elegans* (C. Rankin, personal correspondence) and *Drosophila* (Engel and Wu, 1998). Additionally, genetic alterations of BK channel function has been implicated in different disorders in humans that are associated with short-term habituation deficits in startle, e.g., in schizophrenia (for review see Zhang et al., 2006), mental retardation (Higgins, 2008; Deng et al., 2013), and autism (Laumonnier et al., 2006). Furthermore, the fragile-x related protein, which is impacted in a specific form of autism in humans that is associated with a disruption of habituation has recently shown in mice to directly regulate BK channel activity (Deng et al., 2013). This highly preserved function of BK channels in habituation goes well with the notion of the importance of intact habituation for sensory filtering and higher cognitive function.

BK channels do not seem to play a role in short-term habituation of motivated behavior, and we could not find any evidence for a role in long-term habituation. This supports the idea that there is no universal habituation mechanism, but probably a variety of different mechanisms mediating habituation of different behaviors and at different time scales, as previously proposed (Williams

"fnint-07-00079" — 2013/11/15 — 21:59 — page 6 — #6

et al., 1974, 1975; Davis, 1984; Leaton et al., 1985; Weber et al., 2002; Schmid et al., 2010).

# **ACKNOWLEDGMENTS**

The study was supported by the Canadian Institute for Health Research (CIHR) and the Natural Sciences and Engineering Council (NSERC).

# **AUTHOR CONTRIBUTIONS**

Susanne Schmid conceptualized the study, consulting with Peter Ruth and Peter Pilz. Peter Ruth created the knock-out mice. Peter Pilz, Marei Typlt, and Magdalena Mirkowsi conducted experiments and analyzed data. Marei Typlt ran the stats, wrote the first draft of the manuscript, and made the figures. All authors revised the manuscript. Susanne Schmid finalized the manuscript and submitted it. Marei Typlt and Magdalena Mirkowski have contributed equally to the work.

#### **REFERENCES**


and their role in spike repolarization and regulation of transmitter release. *J. Neurosci.* 21, 9585–9597.


"fnint-07-00079" — 2013/11/15 — 21:59 — page 7 — #7


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 August 2013; accepted: 29 October 2013; published online: 19 November 2013.*

*Citation: Typlt M, Mirkowski M, Azzopardi E, Ruth P, Pilz PKD and Schmid S (2013) Habituation of reflexive and motivated behavior in mice with deficient BK channel function. Front. Integr. Neurosci. 7:79. doi: 10.3389/fnint.2013.00079*

*This article was submitted to the journal Frontiers in Integrative Neuroscience.*

*Copyright © 2013 Typlt, Mirkowski, Azzopardi, Ruth, Pilz and Schmid. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

"fnint-07-00079" — 2013/11/15 — 21:59 — page 8 — #8

# Longterm-habituation of the startle response in mice is stimulus modality, but not context specific

# *Peter K. D. Pilz\*, Stephan W. Arnold , Anja T. Rischawy and Claudia F. Plappert*

*Animal Physiology, Institute of Neurobiology, University of Tuebingen, Tuebingen, Germany*

#### *Edited by:*

*Susanne Schmid, University of Western Ontario, Canada*

#### *Reviewed by:*

*Thomas Preuss, City University of New York, USA Wesley P. Jordan, St. Mary's College of Maryland, USA*

#### *\*Correspondence:*

*Peter K. D. Pilz, Institut fuer Neurobiologie, University of Tuebingen, Auf der Morgenstelle 28, D-72076 Tuebingen, Germany e-mail: peter.pilz@uni-tuebingen.de*

In mice, the specificity of longterm-habituation (LTH) of startle was tested in two experiments. In two strains of mice (C57Bl/6 and C3H) there was pronounced LTH over 10 days of acoustic stimulation in two different contexts of startle measurement. (We found LTH to be greater after stimulation with 14 kHz sine stimuli compared to noise or tactile stimuli). A change of context showed LTH to be independent of context, i.e., startle LTH in mice is a non-associative learning process. In the second experiment, 9 days of acoustic or tactile stimulation were given to C57B/6 mice. Both stimulus modalities produced LTH. When on the 10th day stimuli of the other modality were given, in both cases the long term habituated group showed no lower startle amplitude than a non-stimulated control group. This indicates LTH is stimulus-modality specific. Altogether, our results show that in mice—very similar to rats—LTH of startle is stimulus modality, but not context specific. In addition we found two indications that the LTH action site is on the sensory branch of the startle circuit.

#### **Keywords: long-term habituation, acoustic startle, tactile startle, context, generalization, specificity, auditory pathways**

#### **INTRODUCTION**

Habituation is a sensory filtering process which decreases responses to repetitive stimuli. It is described in a well cited and surprisingly current review by Thompson and Spencer (1966), who characterize habituation by nine points. This characterization was updated 2009 (Rankin et al., 2009), and only one point was to be added: long-term habituation (LTH). Generally, two different forms of habituation are distinguished: firstly, shortterm habituation (STH), which is normally referred to as simply "habituation" and which describes response decreases within one session, with inter-stimulus intervals mostly in the range of seconds or few minutes. Secondly, LTH, describing response decreases between sessions, most often over days (Leaton, 1976). Rankin et al. added LTH as a tenth point to the general habituation concept because this type of learning needs its own neuronal basis, differing from that of STH. LTH in mammals was mainly studied for suppression behaviors and for the acoustic startle response (ASR) (e.g. Leaton, 1976; Jordan et al., 2000). The dependence of LTH on e.g. contextual cues, which is one of the objectives of this study, sometimes differed between response systems studying suppressive behavior or startle (Jordan et al., 2000). Because LTH of the ASR is best understood, both behaviorally and concerning its neural circuitry (e.g. Groves et al., 1974; Leaton, 1976; Jordan and Leaton, 1982, 1983; Leaton and Supple, 1986; Jordan, 1989; Pilz and Leaton, 1999; Jordan et al., 2000), we wanted to address four questions concerning startle LTH in mice:

Firstly, the influence of context on LTH in mice was tested. If context cues are important for LTH in mice, this would point to an associative learning process. On the other hand, if LTH is independent of context, learning should be non-associative (Marlin and Miller, 1981; Jordan et al., 2000). Secondly, longterm changes of STH were studied. This influence of LTH has not been thoroughly examined. Mice are probably well suited for such an interaction study since in this species LTH and STH are slower, i.e., they need more stimuli for a similar decrease, than in rats, perhaps enabling us to describe the interaction better. We wanted to know whether STH remains constant during LTH, or whether it is influenced by LTH. The third goal was to test for stimulus modality specificity of LTH. Jordan and Poore (1998) found no specificity for stimulus frequency of LTH of the ASR; in the same study they found LTH of lick suppression to be frequency dependent. Startle LTH generalized from the training frequency to test stimuli with differing frequencies. Jordan and Poore argue that specificity indeed is one of the original nine characteristics of Thompson and Spencer (1966) for habituation, but that LTH of ASR nevertheless is well suited as paradigm to study habituation. Here we wanted to know whether LTH is modality specific (in more detail than done by Jordan and Leaton, 1982, see Introduction to Experiment 2), in which case at least a part of this original point of Thompson and Spencer would be confirmed.

Our fourth point was to interpret the results together with known facts of the neural startle pathway. This pathway consists of a sensory input branch, a sensory motor interface (giant neurons in the pontine reticular formation) and a motor output branch (Koch, 1999, for details see below). Two of the above results should indicate the action of LTH on this pathway. Recently it was shown that STH is located in the sensory input branch of the startle pathway (Pilz et al., 2004; Vogel and Wagner, 2005). If LTH decreases STH, this may indicate that the sensory input branch is where LTH modulates startle. In addition, it was also shown that the sensory input branches are modality specific, i.e., one input provides selectively auditory information to the sensory motor interface while a different input provides haptic information (Li and Yeomans, 1999; Scott et al., 1999; Simons-Weidenmaier et al., 2006). Thus, if LTH is stimulus modality specific, this would also indicate that the sensory input branch is the action site of LTH.

Two of these questions have already been addressed in rats, where it was shown that context has no influence on LTH learning of startle (Marlin and Miller, 1981; Jordan et al., 2000), while LTH of suppression behaviors is context dependent (Jordan et al., 2000). Since this type of experiment has, to our knowledge, never been repeated in a different species, we wanted to learn more about startle LTH in mice, which are currently quantitatively more important in behavioral research, and which sometimes differ from rats (Frick et al., 2000; Cressant et al., 2007; Snyder et al., 2009; Stranahan, 2011). The modality specificity has been partially addressed by Jordan and Leaton (1982). However, there are several reasons why we should do this experiment again with an experiment designed to answer only this question (see below, Introduction to Experiment 2). Furthermore, because we used three different stimuli to elicit LTH, we managed to find a stimulus that is probably best suited to elicit LTH in mice.

#### **EXPERIMENT 1: CONTEXT SPECIFICITY OF LTH**

One question of the first experiment was whether mice transfer LTH from one context to another. Therefore, in the following we trained mice for LTH to acoustic stimuli in one particular context, and then we tested their reaction to the same stimuli in a different context. We adopted two strains of mice, which possibly differ in their ability to learn contextual cues. When researching the learning of the prepulse inhibition paradigm, Plappert et al. (2006) found differences in amount and velocity of learning between C3H and Bl6 mice strains. These differences were believed to be due to different learning velocities, with Bl6 mice being slower in their ability to learn contextual cues. Indeed, a test of contextual influences partially confirmed this interpretation (Plappert et al., 2006). Thus, we expected either that context cues are unimportant for LTH in mice, similar to rats, or that contextual learning influences LTH. In the latter case, we expected a strain-dependent difference of this influence.

In contrast to the majority of studies about LTH in rats, we administered 100 stimuli per day. This was done in order to study not only LTH to the first responses of each day, but also STH and its change over days.

In addition, we show the results of a pretest, where we did not find reliable LTH to noise stimuli. This is in contrast to the effect of sine stimuli used in the main experiment, which consistently induced LTH. We consider this influence of stimulus quality to be interesting since—to our knowledge—there are no previous publications on the influence of stimulus quality on LTH.

#### **METHODS**

#### *Subjects*

We obtained 48 naïve female C57Bl/6J ("Bl6") mice and 36 naïve female C3H/HeN ("C3H") mice from Harlan Laboratories; 1 C3H was excluded because it did not startle on day one, resulting in 35 C3H. Twenty-four Bl6 mice were measured in a pretest, while the other mice (24 Bl6 plus 36 C3H mice) were measured in Experiment 1. Both strains were 6 weeks old at the beginning of the experiments. The mice were housed in groups of 3 to 4 in standard Macrolon cages containing nesting material under a 12-h light–dark schedule (lights on at 6 am) and received food and tap water *ad libitum*. The cages were in an air-conditioned room with the temperature set at 24◦C, ±1◦C and the humidity held at 60%, ±5%. The mice were adapted to the colony room for 14 days before testing began. Testing took place during the light period. Experiments were approved by the Regional Council of Tuebingen (ZP4/04).

#### *Apparatus*

Startle responses were measured inside a sound attenuated chamber by a movement sensitive piezo accelerometer platform (Startle-Messsystem, Universitaet Tuebingen, Germany) in one of two different contexts (see below). Movement-induced voltage changes were amplified and filtered (Low-Pass: 150 Hz; Piezo-Amp System, Universitaet Tuebingen, Germany) and then digitized with 1 kHz (DAP1200e in a standard personal computer; Microstar, Bellevue, WA). Startle amplitude was calculated as the difference between peak-to-peak voltage during a time window of 50 ms after stimulus onset and peak-to-peak voltage in the 50-ms time window before stimulus onset.

Stimuli and a continuous 45 dB broadband background noise were produced by a digital signal processing controlled system (Elf-Board with Siggen Software; Medav, Uttenreuth, Germany), amplified and emitted by a loudspeaker (Visaton HTM 5.6, Haan, Germany) inside the sound-absorbing chamber. Stimuli for all experiments had an intertrial interval of 15 s.

*Rectangular context 1.* Mice were placed in a rectangular wire mesh test cage (5 × 8.5 × 5.5 cm) with an aluminum floor. The walls of the soundproof chamber (inside measure: 70 × 50 × 40 cm) were covered with dark yellow structured sound absorbing acoustic foam rubber. The chamber was illuminated by a white 5 W cold light bulb. The loudspeaker was located at the side of the test cage.

*Triangular context 2.* Mice were put in triangular test cages with high Plexiglas walls, each wall 11 cm long (height: 30 cm) inside sound absorbing chambers of 45 × 55× 65 cm. The walls of the sound absorbing chamber were covered with bright gray structured sound absorbing foam and vertical bright yellow stripes, illuminated by a white 5 W cold light bulb covered partially (in the direction of the mouse) by a clear bright red plastic sheet. [Since, according to Lall et al. (2010), mice cannot distinguish dark red, illumination was perhaps merely diminished]. The floor of the cage consisted of stainless steel bars (distance 0.7 cm). Below the bars there was a bin with filter paper strips covered with 10% anise oil (freshened before each experiment). The loudspeaker was situated above the test cage.

The two contexts differed with regard to size and geometry of test cage, floor of test cage, structure of test cage walls, color or brightness of illumination, different structures on the walls of the superstructure, odor, direction of sound stimulation, and, probably due to the sound reflecting walls of the triangular context, sensation of the continuous background noise. The triangular context was in a different room. Transportation of the mice (in the test cages) was longer with respect to time and length (3 doors, 30 m), compared to the other context (1 door, 5 m).

#### *Procedure*

Mice were startled on 10 consecutive days in one context. On the following 2 days they were then measured in the same or in a different context. Half of the mice of each strain were startled in the rectangular context in the first 10 days, while the other half was startled in the triangular context. Each half of this half was tested on day 11 in the same, and on day 12 in the different context. The other half was tested on day 11 in the different, on day 12 in the same context.

Each day the mice were adapted for 5 min to the context inside the sound absorbing chambers: they were exposed to the same background noise as during the following stimulation period. They were then exposed to 100 startle stimuli with an interval of 15 s. Startle stimulus consisted of 20 ms white noise 105 dB SPL in the pretest. In experiment 1 it was a 20 ms 14 kHz sine stimulus including 0.4 ms rise and fall times; here the SPL was 105 dB for the Bl6 mice and 100 dB for the C3H mice.

#### *Statistical analysis*

The startle responses were averaged for each day and mouse. Parametric statistics were then calculated with these averages. They were again averaged per strain over mice for days 1–10. Test days 11/12: ASR was averaged for the condition "same" = same context as LTH on day 1–10, and "different" = context differing from LTH context. Statistical analysis was done with JMP (SAS Institute, V. 10). LTH was tested over time by a repeated measures ANOVA on these response values. The Greenhouse-Geisser correction was used because Mauchly'sphericity test was significant (*p <* 0*.*05). LTH was tested on days 11/12 using dependent *t*-tests. In the case of multiple comparisons with day 1 (but not when comparing same context with different context), the *p*-value was Bonferroni corrected (factor 3).

In addition, for the LTH measures for each mouse and day, a relative value (percent of response of day 1) was calculated and the statistics repeated on these percentages. The test results for these relative measures were significant where statistics on absolute values were significant, and vice versa they were not significant for the cases where the tests on absolute measures were nonsignificant; therefore, these test results based on relative values are not reported separately.

STH was calculated as difference between mean first 10 and mean last 40 responses divided by the mean of all responses on the respective day of each mouse. These ratios were averaged over mice (per day), and a linear regression was calculated to test of changes of STH over days.

#### **RESULTS**

### *Pretest: no LTH with noise stimuli*

With noise stimuli as startle stimuli, no significant LTH of the ASR could be observed (data not shown). The average ASR of days 9 + 10 was 96% of the ASR of day 1. This change was not significant [dependent *t(*23*)* = 0*.*45, *p* = 0*.*66].

Since Plappert and Pilz (2005) could show reliable LTH with 14 kHz sine stimuli, this latter stimulus was used in the subsequent experiments. As can be seen below, tonal stimuli reliably induced LTH. Therefore, this pretest shows that acoustic stimulus quality influences LTH.

#### *LTH is not context specific*

In Experiment 1, both strains showed reliable LTH (**Figure 1A**). In Bl6, the average response decreased on day 10 to 53.7% of day 1, in C3H to 48.5%. This decrease was highly significant [Bl6: *F(*4*.*2*,*78*.*9*)* = 6*.*82, *p <* 0*.*0001; C3H: *F(*3*.*9*,*137*)* = 13*.*0, *p <* 0*.*0001].

Half of the mice were tested on day 11 in the different context, the other half in the same context as that of days 1–10. On day 12 all mice were moved to the context different from day 11.

**FIGURE 1 | (A)** Longterm habituation (LTH) to 14 kHz stimuli in two strains of mice (Bl6: C57Bl/6J, C3H: C3H/HeN) over 10 days. Each day comprises the average startle amplitude to 100 stimuli; the decrease was significant in both strains (*p <* 0*.*001). **(B)** C3H, **(C)** Bl6: average startle amplitude on day 1 (same as day 1 in **A**), and on day 11/12 in the same context or on day 11/12 in the different context. The decrease from day 1 to day 11/12 was significant (*p <* 0*.*01), the differences between contexts were not (*p >* 0*.*5). (Bars: s.e.m.; Bl6: *n* = 24, C3H: *n* = 35).

The average ASR on the test days (11/12) was the same, regardless of whether it was measured in the same or in the different context (**Figure 1B**, C3H: ASR in the different context was 97.3% of ASR in same context on day 11/12. **Figure 1C**, Bl6: 108.8%). These small differences were insignificant (both strains: dependent *t <* 0*.*6, *p >* 0*.*5). Again, there was proof of significant LTH since in all cases the ASR was lower on days 11/12 than on day 1, irrespective of the context (dependent *t* ≥ 3*.*42, Bonferroni corrected *p* ≤ 0*.*0074).

#### *Change of short-term habituation over days*

With one small exception, at the end of each daily experiment, the mice startled less than at the beginning. Thus, the curves representing the last 40 responses are consistently below the curves of the first 10 responses (**Figures 2A,B**). In Bl6 mice this difference (representing absolute STH) during each measuring session decelerated over the course of 10 days (insert in **Figure 2B**). Relative STH (i.e., the percent change) also decreased over days; this decrease was significant in Bl6 mice (**Figure 2C** regression of percentages: *r(*8*)* = −0*.*74, *p* = 0*.*014). In contrast, in C3H mice the responses at the beginning and the end of each day decreased in a similar manner during LTH (**Figure 2A**); the relative difference between both measures (**Figure 2C**) did not change significantly [*r(*8*)* = 0*.*43, *p* = 0*.*21]. Thus, in Bl6 the amount of STH decreased over days. This was not the case in C3H, where STH was small from the beginning.

#### **DISCUSSION**

#### *Pretest: no LTH with noise stimuli*

We cannot explain why the mice showed no reliable LTH in the pretest with noise stimuli. Our data with an "LTH" to 96 %, i.e., only 4% decrease, coincide with Azzopardi et al. (2013), who also found only an LTH to 92% using a noise stimulus in the same strain of mice; Typlt et al. (2013) even observed a startle increase over days with noise stimulation. The results in the next experiment shown below with much higher LTH with sine stimuli confirm our own data in previous publications with a reliable LTH using this tonal stimulus (14 kHz, i.e., frequency of best hearing in mice: (Plappert and Pilz, 2005); 10 kHz, i.e., frequency of best hearing in rats: Pilz and Leaton, 1999). In contrast to these findings, Schmid et al. (2011) found a reliable LTH (about 50%) with noise stimuli in hybrid mice with a mixed C57Bl/6 × 129S6 genetic background. In a shorter test (over 5 days), we found no LTH in C3H mice to noise stimuli; however, we also found reliable LTH in C3H when tested over a longer period with 250 daily noise stimuli (unpublished data). Thus, while noise stimuli often result in no or almost no LTH, obviously a 14 kHz stimulus is a good stimulus to elicit LTH in mice.

We can only hypothesize why noise stimuli produce less LTH. One reason could be that noise stimuli are not as constant as tonal stimuli (neither in amplitude nor in frequency), and thus perhaps more difficult to learn. However, since Jordan and Poore (1998) showed no influence of acoustic stimulus frequency on LTH, this explanation is not convincing. Another reason might be that noise stimuli elicit long-term sensitization, thereby counteracting LTH, a process shown by Borszcz et al. (1989). This type of effect was reliably found in mice by Typlt et al. (2013). If this is the case, we

**FIGURE 2 | Long-term habituation of short-term changes. (A)** (C3H), **(B)** (Bl6): Green lines are averages over the first 10 responses of each day; blue is the mean of the last 40 responses. For comparison, mean first responses are also shown (dotted lines). Inset in B (Bl6): short term habituation on day 1, 6, and 10 (from top to bottom; blocks of 10 responses). Bars: s.e.m.; for clarity, SEMs of 1st responses are not shown; they were always larger than SEMs of first 10 responses (mean factor: 1.87 in C3H, 1.48 in Bl6). **(C)** Short-term habituation (STH) on each day shown as mean percent change between mean first 10 and mean last 40 responses. The decrease of these differences over days was significant in Bl6 (*p <* 0*.*02), but not in C3H (*p >* 0*.*2).

still cannot explain why a noise stimulus should produce longterm sensitization, and a tonal stimulus should not.

To summarize the unpredicted stimulus dependency of LTH: in the following experiments and in future experiments we use 14 kHz stimuli to reliably elicit LTH in mice.

#### *Main experiment: long-term habituation is not context specific*

Both strains of mice transferred the LTH from one context to the other context (**Figures 1B,C**). The average startle responses were almost identical in the two different contexts. Therefore, we must conclude that context plays no role for LTH in mice.

This outcome confirms the findings of Marlin and Miller (1981) and Jordan et al. (2000) that LTH is not context specific in rats. Since we have now demonstrated that the same is true for mice, we believe we can apply this notion to rodents in general. In this group of animals it is obvious that startle LTH does not depend on association with context cues, confirming that LTH is a non-associative learning process.

Jordan et al. (2000) showed that this non-associative aspect of LTH is only true for startle LTH, and not for other long term habituation mechanisms for other paradigms. They discuss associative vs. non-associative models for LTH. Because they drew paradigm dependent conclusions, we refer the reader to this publication for discussion of the non-associative vs. associative nature of LTH in general. Their use of diverse response systems also offered the advantage of controlling the effect of background cue differences: tests of lick suppression, parallel to ASR testing, showed that the rats were able to differentiate between the contexts when this measure was evaluated.

It could be argued that in our study the mice did not learn the context cues available (Marlin and Miller, 1981). However, since we changed several cues in several sensory modalities simultaneously, for the purpose of startle testing in the lab, there is no measurable transfer of contextual learning between contexts.

#### *Influence of long-term on short-term habituation*

While in C3H mice the STH was small from the beginning, in Bl6 it was initially large and decreased over days. This leads to a smaller LTH in Bl6 compared to C3H, when all responses are used to characterize the daily ASR (**Figure 1A**). If only the first 10 responses are considered, LTH is much more similar between strains (compare "first 10" curves in **Figures 2A,B**). Since there was a reliable decrease of STH over days in Bl6, LTH interacted with this learning process. For the purpose of this discussion we evaluated data of another experiment in our lab, which did not strive to measure habituation and used more than 200 daily stimuli. In C3H we also found a significant STH on day 1; in this case STH also decreased significantly over days, paralleling the course of LTH (unpublished data).

We want to speculate about where LTH and STH interact in the startle pathway. Until now, all studies have suggested that the neuronal structure mediating LTH is an extrinsic pathway which modulates the startle pathway. Lesions to the mesencephalic reticular formation (MRF) (Jordan and Leaton, 1982; Jordan, 1989) or vermis of the cerebellum (Leaton and Supple, 1986, 1991; Lopiano et al., 1990) attenuate or eliminate LTH. Lesions of the inferior colliculus did not alter LTH (Jordan and Leaton, 1983), as did complete decerebration at the level of the mesencephalon (Leaton et al., 1985). Recently the involvement of the cerebellum in LTH has been confirmed in humans (Timmann et al., 1998; Maschke et al., 2000; Pissiota et al., 2002; Frings et al., 2006). Lesion of the MRF eliminates the long-term decrease of startle not only before, but also after LTH training (Jordan, 1989). Therefore, LTH is not a morphological change of e.g. synapses within the startle pathway, but LTH must be an extrinsic chronic modulation acting on this pathway. It must be chronic, since even fast EMG components of startle are decreased when they experience LTH [Poore and Jordan (1992); Jordan et al. (1993): cited in Jordan and Poore (1998)]. Thus, LTH chronically modulates the startle pathway at an unknown action site.

The auditory input from ear and acoustic nerve is common for auditory startle pathway, hearing and prepulse inhibition circuit (Plappert and Pilz, 2013). The neurons of the acoustic nerve are primary sensory neurons of the auditory startle circuit, followed by startle pathway specific secondary sensory neurons (Koch et al., 1992; Lingenhöhl and Friauf, 1992; Lee et al., 1996). The latter project onto giant neurons in the PnC (Koch et al., 1992; Lingenhöhl and Friauf, 1992), which are the sensory motor interface of startle. The motor output part begins as direct or indirect efferences of the giant neurons to motoneurons (Davis et al., 1982; Lingenhöhl and Friauf, 1994; Yeomans and Frankland, 1996). We believe we can exclude LTH acting at the level of the ear or acoustic nerve based on two findings: first, hearing is constant at this age of mice and at the frequency range used (Ehret, 1974); second, if processing up to the primary auditory neurons would be depressed by LTH, prepulse inhibition to acoustic prepulses should decrease during LTH, which is not the case (Plappert and Pilz, 2005). We also believe, however with much less confidence, that we can exclude LTH acting on the motor branch of the startle circuit. Due to the constant extrinsic inhibition by LTH, we would expect the decrease at the end of a day to be similar to that at the beginning, i.e., the percent STH should remain roughly constant during LTH (which is not the case). Thus, we are left with the startle specific secondary neurons or the giant PnC neurons. Several mechanisms may be at work here. The first possibility is the direct interaction of LTH with STH. Since STH of startle is a process situated in the sensory input branch (Pilz et al., 2004; Simons-Weidenmaier et al., 2006; Schmid et al., 2011), this would also place LTH in this branch. A second possibility would be that LTH acts selectively on neurons or nuclei processing higher auditory input. Meloni and Davis (1998), for example, have shown that the dorsal cochlear nucleus (DCN) contributes to a high intensity component of the acoustic startle reflex. LTH acting on this nucleus would result in the patterns we observed: LTH and a decrease of STH since the dynamic range of startle is restricted to a lower input range. Another possibility is that LTH selectively inhibits large caliber PnC-neurons, which would also result in a similar pattern. More complicated schemes may also come to mind, e.g. a decrease of STH over days independent of LTH, in which case no part of the pathway can be excluded as an action site of LTH. However, since only the first two mechanisms discussed here are based on current literature (and are more simple in the sense of Ockham's razor), we believe that the LTH action probably takes place on the sensory side of the startle pathway.

# **EXPERIMENT 2: STIMULUS MODALITY SPECIFICITY OF LTH**

In this experiment we wanted to know whether LTH is specific for the modality of the learned stimulus. Stimulus specificity and stimulus generalization are important characteristics of habituation (Thompson and Spencer, 1966; Rankin et al., 2009). STH of startle is stimulus modality specific. If startle is short term habituated to one modality (tactile or acoustic), there is no generalization of this habituation to a different modality (Pilz et al., 2004; Vogel and Wagner, 2005). There are no such thorough analyses for LTH. In rats, daily acoustic stimuli elicited more LTH in controls than in rats with a lesion of the mesencephalic reticular formation (MRF; Jordan and Leaton, 1982). This difference between lesioned and control rats was not extended to tactile stimulation. While this suggests that LTH is modality specific, there are still some discussion points. The MRF lesion could have changed LTH specifically to acoustic stimuli. This is of importance since Jordan and Leaton tested specificity "only" from acoustically habituated rats to tactile stimulation.

In addition, in Jordan and Leaton (1982) the tactile stimulus had an acoustic component of 75 dB, which by itself did not elicit startle. Since the controls had lower startle amplitude to their complex tactile-acoustic stimulus, there might have been LTH in the controls to the acoustic component. If so, there might have been a general response amplitude difference between their tactile and their acoustic stimulation. As Jordan and Poore (1998) argue, new stimuli eliciting higher startle amplitudes are not easy to interpret in the light of LTH. When Jordan and Poore (1998) discussed the switch of acoustic quality in Jordan and Leaton (1982), they state critically: "However, this study did not attempt to equate stimulus intensities, and no animals were habituated to the noise stimulus and then switched to the pure tone." In this sense, we wanted to look again into modality specificity, with tactile stimuli without acoustic artifact (using a silencer, Pilz et al., 2004), with stimuli roughly equaling stimulus intensities (the intensities of acoustic and tactile stimulation produced crossing LTH curves), and with animals trained to both stimulus modalities.

Jordan and Poore (1998) found no frequency specificity of LTH. LTH was stable if the stimulus frequency was changed from 10 to 22 kHz or vice versa, and if the same was done for 10 and 35 kHz. We thus know there is maximal generalization within this modality (Jordan and Poore, 1998), while there seems to be no generalization over modalities (Jordan and Leaton, 1982). Since our criticism of Jordan and Leaton (1982) is farfetched, our expectation was to confirm their result of stimulus specificity of LTH. If so, our results would extend this feature of LTH to mice, to unlesioned controls, and to cross-habituation in both directions. For this purpose, groups of mice were long-term habituated to either tactile or acoustic stimuli over a period of 9 days (with control groups for the respective time and background noise condition in the startle apparatus). On the 10th day they were tested using the different modality.

#### **METHODS**

#### *Subjects*

Subjects were 16 female and 28 male naïve Bl6 mice. They were divided into four groups. In both control groups there were 4 female and 6 male mice. There were 4 female and 8 male mice in the groups exposed to stimuli during training. Age, keeping, supplier, etc. were the same as in experiment 1.

#### *Apparatus*

The same apparatus and measures were used for acoustic stimulation as in Experiment 1. The only difference was the changed test cage (see below).

The apparatus for tactile stimulation is described and discussed in detail in Pilz et al. (2004); changes are described in the following. Tactile stimuli were airpuffs of 100 Pascal, measured at the center of the cage; the air pressure before the air valve solenoid was 2 bar = 29 psi. The airpuffs were delivered through a PVC tube centered on the side of the test cage, 7.5 cm from the center of the cage. The end of the round tube was compressed to an inner ellipsoid diameter of 1*.*2 × 0*.*25 cm. The tube was directed toward a cage with the same size as in experiment 1, with the single difference that it was elevated above the measuring platform by 3.5 cm by means of four stilts. The tube nozzle was at the same height, but directed to the middle of the cage; thus the air was directed slightly away from the measuring platform, which minimized the risk that tactile artifacts could be falsely measured as startle responses.

The airpuff characteristics were measured using a 1-inch (2.54-cm) microphone (Model 4145; Bruel and Kjaer, Naerum, Denmark). The airpuffs had a duration of 20 ms plus rise and decay times each lasting 8 ms and a pressure of 100 Pa. To reduce noise generated by the air valve solenoid, the air passed through a "silencer." The silencer was a glass cylinder (diameter 5.3 cm, height 8.2 cm, volume 168 cubic cm) sealed with a rubber stopper. The rubber stopper had two holes, one for air inlet and the other for air outlet. The cylinder contained sound absorbing rubber foam (thickness 4.3 cm, on the side opposite the rubber stopper). To mask the sound of the airpuff itself, which had mainly low frequency components (Pilz et al., 2004), we performed all testing of the tactile startle response with background noise containing frequencies between 250 Hz and 20 kHz, with maximum intensity at 2 kHz. The noise was produced by a digital signal processing controlled system (Elf-Board with Siggen software; Medav, Uttenreuth, Germany), amplified and emitted by a second loudspeaker (Craaft HT 1640; Solton Music, Pocking, Germany) inside the soundabsorbing chamber. At a background noise level of 93 dB SPL RMS, the acoustic artifact of the tactile stimuli was completely masked.

Acoustic (14 kHz, 105 dB SPL, see Experiment 1) and tactile stimuli for all experiments had an intertrial interval of 15 s.

#### *Procedure*

As in Experiment 1, mice were adapted for 5 min daily to the startle apparatus. During each daily session the background noise was constant: 45 dB for the acoustic startle measures and 93 dB for the tactile startle measures. One-hundred stimuli were then given.

The four groups were:

Group acoustic: 9 days acoustic stimulation, then 1 day tactile stimulation.

Group acoustic-control: 9 days 45 dB background noise without stimulation, then 1 day with tactile stimulation.

Group tactile: 9 days tactile stimuli, then 1 day with acoustic stimuli.

Group tactile-control: 9 days 93 dB background noise without stimulation, then 1 day with acoustic stimuli.

#### *Statistical analysis*

Statistics were the same as in Experiment 1, with the following exceptions: since Mauchly's test was not significant, the uncorrected *F*-tests of the repeated measures ANOVA are reported for LTH (day and gender were statistically tested by two-factor ANOVAs).

#### **RESULTS**

The mice displayed strong LTH to acoustic stimuli on 9 days (mean day 9 = 29.6% of day 1; **Figure 3A**). The startle response to tactile stimulus on test day 10 was even slightly higher in the acoustic (i.e., LTH) group, compared to the acoustic-control group. There was no statistical difference between pretreatment groups. A two-factor ANOVA revealed a trend for a gender effect [*F(*1*,* <sup>18</sup>*)* = 3*.*81, *p* = 0*.*067], but no interaction [*F(*1*,* <sup>18</sup>*)* = 1*.*53, *p* = 0*.*23]. Most importantly, there was no significant effect of prestimulation on the test day [*F(*1*,* <sup>18</sup>*) <* 1]. Pairwise comparisons showed that tactile startle on day T10 (acoustic and acoustic control groups, **Figure 3A**) was not different from day T1 (tactile group, **Figure 3B**; uncorrected *t*-tests, *t* ≤ 1*.*1, *p* ≥ 0*.*27), but differed at least partly from day T9 [acoustic control: *t(*20*)* = 1*.*9, *p* = 0*.*073; acoustically stimulated group: *t(*22*)* = 2*.*8, *p* = 0*.*010].

There was only a weak LTH to tactile stimuli [mean day 9 = 83.8% of day 1; **Figure 3B**; *F(*8*,* <sup>80</sup>*)* = 2*.*33, *p* = 0*.*026; gender: *F <* 1; interaction gender × day: *F(*8*,* <sup>80</sup>*)* = 1*.*18, *p* = 0*.*32]. Mean responses on test day 10 to acoustic stimuli were even slightly higher in the tactile (LTH) group compared to the tactilecontrol group. The two-factor ANOVA yielded no effect of gender [gender, interaction gender × prestimulation: *F(*1*,*18*) <* 1*.*8, *p >* 0*.*20], and no effect of prestimulation [*F(*1*,* <sup>18</sup>*)* = 2*.*32, *p* = 0*.*15]. Pairwise comparisons showed that acoustic startle on day A10

**FIGURE 3 | (A)** Mice were startled with tactile stimuli on day 10 (T10) after 9 days (A1–9) of either 100 acoustic startle stimuli daily (group "acoustic"), or 9 days without acoustic stimulation ("control"). **(B)** Mice were startled on day 10 (A10) with acoustic startle stimuli, after 9 days (T1–9) with either 100 daily tactile startle stimuli ("tactile") or without stimulation ("control"). The amplitude decrease during the first 9 days was significant in both stimulated groups (*p <* 0*.*05). There was no significant difference between the previously stimulated and the unstimulated groups on day 10 (*p >* 0*.*1. Bars: s.e.m.; *n* = 44).

(tactile and tactile control groups, **Figure 3B**) was not different from day A1 (acoustic group, **Figure 3A**; uncorrected *t*-tests, *t* = 1*.*59, *p* = 0*.*13), but differed from day A9 (*t* = 2*.*51, *p* = 0*.*020).

#### **DISCUSSION**

#### *No cross-habituation of LTH*

There was clearly no cross-habituation if LTH was elicited by one stimulus modality, and then startle was tested by another stimulus modality. For acoustic LTH, i.e., 9 days of daily acoustic stimulation, this was very convincing since there was a large LTH to this modality of about 70% response decrease. If cross-habituation had occurred, we would expect a lower startle response to tactile stimuli after the acoustic LTH. However, the tactile response was the same in mice with acoustic LTH as in mice without acoustic training.

Tactile LTH training of 9 days produced a response decrease of only 16%. It is therefore not particularly convincing that no cross-habituation occurred. Nevertheless, the mice tested after this procedure did not startle less when subjected to acoustic stimuli than mice without tactile training.

Our results partially confirmed those of Jordan and Leaton (1982). They found LTH to acoustic stimuli to be modality specific in rats. There daily acoustic stimuli elicited more LTH in controls than in rats with a lesion of the MRF. This difference between lesioned and control rats was not extended to tactile stimulation. Since the MRF-lesion could have changed LTH specifically to acoustic stimuli, our results extend their finding not only to mice, but also to unlesioned controls.

While the decrease to tactile stimuli here was only 16%, in Plappert and Pilz (2005) it was 34% under comparable parameters. Borszcz et al. (1989) showed long-term sensitization counteracting LTH. Stimulus intensity for tactile stimuli was slightly higher than in the study of Plappert and Pilz (2005). This may have produced higher sensitization, and therefore smaller LTH. Absolute amplitudes elicited by acoustic and tactile stimuli were roughly the same (if averaged over all days; **Figure 3**). Hence, if our tactile stimuli induced sensitization, this should be specific for this modality, since it cannot be due to higher startle responses elicited by this stimulus. (Indeed, on day 1 tactile startle was lower than acoustic startle). Because in rats LTH to tactile stimuli is also relatively small compared to LTH to tonal stimuli (Jordan and Leaton, 1982), this might be characteristic for this modality.

However, another type of sensitization could have influenced the results. During acoustic stimulation days, the background noise SPL was 45 dB, while on days with tactile stimulation it was 93 dB. The 93 dB background noise was necessary to completely mask the acoustic artifacts produced by the airpuffs used for tactile stimulation, and it might have sensitized the mice. For this reason each experimental group had a control group with precisely the same background noise experience on the same days. E.g. the group which was subjected to acoustic stimuli (and thus 45 dB background noise) on days 1–9 (**Figure 3A**) had a control group of mice subjected to the same noise on days 1–9. When on day 10 the two groups were subjected to tactile stimuli for the first time, they also heard the higher background noise SPL for the first time, which might have influenced the results. However, the tactile startle amplitudes on this day were not different from those of the mice with tactile stimulation on day 1 (**Figure 3B**, T1), but higher than that of the same mice on day 9 (**Figure 3**, T9). Furthermore, the results were identical for the other direction of cross-habituation. Thus, since the quantitative and statistical results are as expected if cross-habituation is absent, and since the amplitudes of tactile and acoustic startle from the beginning were different, and furthermore, since the LTH to acoustic and tactile startle were different, we find it unlikely that these exact results are due to background noise changes. We cannot rule out influences of the background noise differences on these results; however, we believe the results indicate absence of cross-modality in LTH.

Testing another change of stimulus characteristics, Jordan and Poore (1998) found no specificity for stimulus frequency. LTH remained constant if stimulus frequency was changed by one or two octaves. The lack of generalization was restricted to LTH of startle, but could be measured in other paradigms. We feel that Jordan and Poore correctly argue that response latency of startle is much shorter than in the other paradigms. Typically latency of startle in rodents can be as short as 7 ms (Plappert and Pilz, 2013), electromyographically even below 6 ms (Caeser et al., 1989). This is probably too short for frequency specific processing. Therefore, although stimulus generalization, one of the nine (Thompson and Spencer, 1966) respectively 10 (Rankin et al., 2009) key characteristics of habituation, is absent inside the auditory stimulus modality, it is demonstrable between stimulus modalities.

Although LTH was different on training days 1–9 to acoustic and tactile stimuli, in both cases the startle elicited afterwards by the different modality was never lower in the LTH-trained mice than in the controls. Thus, we conclude that LTH in mice is stimulus modality specific. Furthermore, because this confirms the conclusions found in literature on rats, we believe that this specificity applies to rodents in general.

#### *Assumed neuronal action of LTH on startle pathway*

As already pointed out above, the neuronal pathway of acoustic startle consists of primary and secondary auditory neurons projecting onto giant neurons in the PnC, which are the sensorimotor interface and project themselves onto motoneurons (Plappert and Pilz, 2013). Tactile input from the face (i.e., relayed by the sensory trigeminal nerve) also comes from secondary sensory neurons to the giant neurons (Li and Yeomans, 1999; Scott et al., 1999; Simons-Weidenmaier et al., 2006). Whether and how haptic input from the body is relayed to the giant neurons is unclear [discussed in Pilz et al. (2004)]. Current discussions assume that they are also connected to giant neurons (e.g. Simons-Weidenmaier et al., 2006).

STH of startle is modality specific (similar to LTH shown here). Startle decrease elicited by acoustic stimuli is not transferred to the tactile modality; neither is the startle decrease elicited by tactile stimuli transferred to the acoustic modality (Pilz et al., 2004; Vogel and Wagner, 2005; Ponce et al., 2011). Since the sensory inputs to the common PnC interface are distinct for auditory and haptic information, and since several publications propose that the pathway from PnC neurons onwards is identical for both sensory modalities (Li and Yeomans, 1999; Scott et al., 1999; Simons-Weidenmaier et al., 2006), STH must be situated in the respective sensory branches. Indeed, it has been shown that this process probably takes place in the synapse of secondary sensory neurons on giant PnC neurons (Simons-Weidenmaier et al., 2006).

Here we demonstrated that LTH is also stimulus modality specific. With the same logic as for STH, LTH also should act on the respective sensory branches of the startle pathway. We cannot exclude unknown additional motor branches for both sensory modalities studied, as found for vestibular startle (Li et al., 2001). However, since the current knowledge regarding startle circuitry does not (yet) include this possibility for acoustic and tactile startle, we are confident that LTH acts prior to the motor part of the pathway of this response system. In addition, after our findings in Experiment 1, this is a second indication for a sensory location of neuronal LTH action.

This neural implementation would restrict LTH generalization. As pointed out above, startle latency is probably too short for stimulus specificity inside one modality. However, by acting differentially on sensory branches, some specificity of LTH would be preserved, which would prevent unnecessary inhibition of modality-different stimulation.

# **GENERAL DISCUSSION**

#### **STARTLE LTH DEPENDS ON STIMULUS TYPE USED**

LTH differed for different types of stimuli. In both experiments there was a strong LTH to 14 kHz tonal startle stimuli. LTH was significant but much weaker regarding tactile stimuli and almost non-existent to noise stimuli in our pretest. As already discussed in Experiment 1, the tonal stimuli were exactly the same each time they were presented, and thus might be best suited to elicit LTH. Noise stimuli are not constant in course of amplitude or frequency, which might interfere with their ability to elicit LTH. Since mice move in the test cages, the airpuffs hit different parts of the body. Pilz et al. (2004) show that this movement did not influence STH in their study. Nevertheless, haptic stimulation of different parts of the skin may also prove to be an inconsistent stimulus with respect to LTH. However, Jordan and Poore (1998) found evidence that refutes this explanation, i.e., they found LTH to explicitly non-constant stimuli. In their experiments, LTH was independent of stimulus frequency during testing compared to training. Whatever the correct explanation, tonal stimuli, such as 14 kHz, which is in the best hearing range of mice (Ehret, 1983), seem to be most useful for LTH training.

#### **STARTLE LTH IS A NON-ASSOCIATIVE LEARNING PARADIGM**

The strongest suggestion that startle LTH is non-associative can be derived from experiment 1, showing that LTH is constant after context change, thereby confirming original literature (Marlin and Miller, 1981; Jordan et al., 2000). Since Jordan et al. (2000) also show that for other response systems LTH is associative, the non-associative nature of LTH shown here is restricted to the startle response system. If this process is non-associative, it must depend solely on stimulus characteristics. This obviously is the case, since LTH depends on the stimulus modality used (Experiment 2). If LTH was elicited by acoustic or tactile stimuli, and then stimulus modality was changed (to tactile or acoustic, respectively), there was no transfer of LTH from one to the other modality. Thus, taken together, all data suggest that startle LTH in mice is non-associative, as discussed previously for rats (Marlin and Miller, 1981; Leaton and Supple, 1986; Jordan and Poore, 1998; Jordan et al., 2000).

#### **PROPOSED NEURONAL ACTION OF LTH ON STARTLE PATHWAY**

Two outcomes suggest the same location of neural action of LTH on the startle pathway. In Experiment 1, LTH was shown to possibly have a strong diminishing effect not only on startle amplitude, but also on STH of startle (in the one strain exhibiting reliable STH); to date, STH has only been found in the sensory branches of the startle pathway (Simons-Weidenmaier et al., 2006). In the second experiment, LTH proved to be sensory modality specific. This also indicates that the modality specific neuronal input branches of the startle pathway are the location of LTH action.

#### **SUMMARY**

A context change did not disrupt startle LTH; neither was there a transfer of LTH from one stimulus modality to the other (tactile to acoustic or vice versa). So, similar to previous data from rats, our results indicate that LTH in mice is a non-associative stimulus modality specific learning paradigm. Results from both experiments together with data from the literature suggest that the neuronal action of LTH is a chronic inhibition aiming at the modality specific sensory input branches of the startle pathway. Moreover, we found the best LTH to tonal startle stimuli compared to noise pulses or tactile airpuffs.

# **AUTHORS CONTRIBUTIONS**

Peter K. D. Pilz planned the experiments, tested data statistically and wrote the manuscript. Stephan W. Arnold and Anja T. Rischawy conducted the experiments, including optimization of the contexts and tactile stimulation, and compiled part of the descriptive statistics as well. Claudia F. Plappert planned experiments and contributed to the discussion and the manuscript. There was no conflict of interests for any author.

#### **ACKNOWLEDGMENTS**

This work was supported by the Deutsche Forschungsgemeinschaft (Grant PI450/2-1 to Peter Pilz). We thank Donna Blagg for correcting our English.

# **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 23 August 2013; accepted: 16 December 2013; published online: 09 January 2014.*

*Citation: Pilz PKD, Arnold SW, Rischawy AT and Plappert CF (2014) Longtermhabituation of the startle response in mice is stimulus modality, but not context specific. Front. Integr. Neurosci. 7:103. doi: 10.3389/fnint.2013.00103*

*This article was submitted to the journal Frontiers in Integrative Neuroscience.*

*Copyright © 2014 Pilz, Arnold, Rischawy and Plappert. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Saliency mapping in the optic tectum and its relationship to habituation

# *Arkadeb Dutta andYoram Gutfreund\**

*Rappaport Family Institute for Research in the Medical Sciences, Ruth and Bruce Rappaport Faculty of Medicine, Technion – Israel Institute of Technology, Haifa, Israel*

#### *Edited by:*

*Susanne Schmid, University of Western Ontario, Canada*

#### *Reviewed by:*

*Elisabetta Ladavas, University of Bologna, Italy Manuel S. Malmierca, University of Salamanca, Spain Hermann Wagner, Rheinisch-Westfaelische Technische Hochschule Aachen University, Germany*

#### *\*Correspondence:*

*Yoram Gutfreund, Rappaport Family Institute for Research in the Medical Sciences, Ruth and Bruce Rappaport Faculty of Medicine, Technion – Israel Institute of Technology, Efron 1 Street, Haifa, Israel e-mail: yoramg@tx.technion.ac.il*

Habituation of the orienting response has long served as a model system for studying fundamental psychological phenomena such as learning, attention, decisions, and surprise. In this article, we review an emerging hypothesis that the evolutionary role of the superior colliculus (SC) in mammals or its homolog in birds, the optic tectum (OT), is to select the most salient target and send this information to the appropriate brain regions to control the body and brain orienting responses. Recent studies have begun to reveal mechanisms of how saliency is computed in the OT/SC, demonstrating a striking similarity between mammals and birds. The saliency of a target can be determined by how different it is from the surrounding objects, by how different it is from its history (that is habituation) and by how relevant it is for the task at hand. Here, we will first review evidence, mostly from primates and barn owls, that all three types of saliency computations are linked in the OT/SC. We will then focus more on neural adaptation in the OT and its possible link to temporal saliency and habituation.

**Keywords: habituation, saliency map, optic tectum, superior colliculus, spatial attention, barn owl, orienting response**

# **INTRODUCTION: HABITUATION AND SALIENCY MAPPING**

An animal can respond to only one stimulus source at a time, even though its sensory systems are bombarded by information arriving from multiple sources. This unequal relationship between response and stimulus numbers explains the evolvement of brain mechanisms to identify and select the most appropriate stimulus for behavioral manifestation. The process, commonly known as selective attention, saliency mapping, or sensory gating (Itti and Koch, 2001; Krauzlis et al., 2013), is a fundamental characteristic of human and animal behavior. Failing to choose the appropriate stimulus severely disrupts normal behavior, as can be seen in the various attention deficits disorders (Gitelman, 2003; Booth et al., 2005).

How can an animal decide if a stimulus is behaviorally relevant? What are the sensory cues and to what extent are they general across species? A dog among cats, a brown object among orange objects, or a pure tone succeeding a long period of broadband noise are all conspicuous stimuli. Such stimuli are perceived as salient and consequently behaviorally privileged as they give rise to attentional capture (Posner, 1980; Tiitinen et al., 1994), enhanced autonomic responses (Weisbard and Graham, 1971; Bala and Takahashi, 2000; Zimmer, 2006), and enhanced neural responses (Naatanen, 1995; Gottlieb et al., 1998; Ulanovsky et al., 2003; Reches and Gutfreund, 2008). Thus, a general rule is that a stimulus that is out-of-ordinary (unexpected) is salient and has a higher probability to induce responses. In nature, an out-of-ordinary event may come as a warning to prey to evade danger or an opportunity for food in the case of predators. Therefore, rapid detection of such unexpected events is fundamental for survival. A stimulus can be out-of-ordinary in space, as in the first two examples above, or out-of-ordinary in time, as in the third example above. The process of detecting stimuli that are out-of-ordinary in space is called stimulus competition, spatial saliency mapping, or camouflage breaking (Itti and Koch, 2000; Knudsen, 2007). The process of detecting stimuli that are out-ofordinary in time is called temporal saliency mapping, deviance detection, or change detection (Nelken and Ulanovsky, 2007; Gutfreund, 2012).

Imagine a dog resting in the center of a noisy living room, seemingly ignoring the loud noise of kids, and music playing but immediately raising its head to the faint sound of the door knob turning. Note that the event of the door knob turning is not louder than the background noise. This example demonstrates a major aspect of saliency mapping; that the saliency of a stimulus is not determined by its physical strength but by its relationship with the environment, or, in other words, by its context. The door knob turning is a salient event mainly because it breaks the regularity of the background and is therefore unexpected.

In the case of temporal saliency, as in the door knob example, an initial phase of learning and memorizing the regularity of the background must take place so that an incoming stimulus can be categorized as either background or deviant. Responses to stimuli matching the background are suppressed while responses to deviants from the background are not. **Figure 1** illustrates a conceptual model for temporal saliency mapping. Interestingly, a similar concept was used to model habituation of the orienting reflex (Sokolov, 1963; Siddle, 1991). Thus, there seems to be a considerable overlap between habituation and saliency mapping.

"fnint-08-00001" — 2014/1/14 — 15:38 — page 1 — #1

Habituation is considered the most basic form of learning, existing in all animals (Sokolov, 1963; Thompson and Spencer, 1966; Thompson et al., 1972; Barry, 2009; Thompson, 2009). Although habituation has been described and studied in detail for decades, most of the previous studies on habituation were separated from studies on saliency mapping and attention (Dukewich, 2009). In this review, we aim at emphasizing the close relationship prevailing between habituation and saliency mapping and the likelihood that there is a considerable overlap between the neural mechanisms of the two processes.

In addition to habituation, associative learning may also be involved in saliency mapping (Anderson, 2013). It is obvious that the successful selection of stimuli cannot be based on external factors alone. Selection of a stimulus must be guided by a combination of external factors, such as stimulus intensity, stimulus history, spatial context, cross-modal interactions, etc., and internal factors, such as cognitive biases, behavioral tasks, reward history, motivations, etc. (Fecteau and Munoz, 2006). For example, when searching for a friend in crowd, a memorized knowledge such as shirt color or hair type biases the perceived saliency of stimuli. Such information about the relevance of the stimulus for the task at hand is called "top-down" information as opposed to information about the sensory aspects of the stimulus, which is called "bottom-up" information. Somewhere in the brain, top-down information must be integrated with bottom-up information to determine the saliency of the event (Fecteau and Munoz, 2006). Where in the brain this integration happens and what are the mechanisms involved are open questions. It is possible that top-down information can modulate the internal representation of the background created by the bottom-up stream (illustrated in **Figure 1**). This hypothesis is attractive as it implies that cognitive factors may influence stimulus selection by recruiting the same brain structures that are involved in temporal saliency and habituation. We will hypothesize here and show evidence that brain areas or networks that select stimuli for behavior are likely to show neural correlates of habituation. The brain structure that we will focus on is the optic tectum (OT), also known as the superior colliculus (SC) in

"fnint-08-00001" — 2014/1/14 — 15:38 — page 2 — #2

mammalian species. We will first review evidence supporting a role of the OT/SC in both spatial and temporal saliency mapping and then focus more on temporal saliency mapping and its relation to habituation (for more comprehensive reviews on spatial saliency mapping see Itti and Koch, 2001; Mysore and Knudsen, 2011b).

# **SALIENCY MAPPING IN THE OT/SC**

All vertebrates possess a specialized brain system responsible for orienting the body toward stimuli of interest. This system, known as the gaze control system (also referred to as the oculomotor system), involves a number of midbrain and forebrain areas (see scheme of basic avian circuitry in **Figure 2**). The OT/SC is a midbrain structure serving as a critical hub in the gaze control system (Boehnke and Munoz, 2008). This structure is arguably one of the most phylogenetically conservative structures in the brain (Gaither and Stein, 1979; Shimizu and Karten, 1993; Luksch, 2003) and is considered homolog in all vertebrate species (Butler and Hodos, 2005; Maximino, 2008).

Both mammalian SC and avian OT contain a mapped representation of space. This map is multimodal with neurons responding to auditory, visual, and somatosensory signals (Knudsen, 1982; King and Palmer, 1985; Stein and Meredith, 1993; Nodal et al., 2005). The sensory map of space is superimposed over a motor

**FIGURE 2 | Illustration showing a selected part of the barn owl's auditory pathway with the OT as a hub.** Gray boxes indicate structures suggested to be related to gaze and attention control. The red outline indicates structures where SSA was reported; the blue outline indicates areas where long-term adaptation was reported (habituation-like responses); the yellow outline indicates structures where multisensory (visual/auditory) neurons were reported. Double-headed arrows indicate reciprocal connections. Abbreviations of names of structures appear within each box. The abbreviations of names of the equivalent mammalian structures are in parentheses: E, entopallium; AGF, arcopallium gaze fields; FEF, frontal eye fields; nRt, nucleus rotundus; OTs, superficial layers of the OT (layers 1–10); OTi/s, intermediate and deep layers of the OT (layers 11–15); IC, inferior colliculus; Ipc, nucleus isthmi pars parvocellularis; Imc, nucleus isthmi pars magnocellularis; nOv, nucleus ovoidalis; MGN, medial geniculate body; A1, auditory cortex 1.

map controlling gaze shifts (McHaffie and Stein, 1982; Sparks, 1986; Herrero et al., 1998; King, 2004). Hence, the OT/SC is known primarily as a gaze control center serving to translate sensory signals to eye and head movements. However, a careful examination of the anatomy with a detailed electrophysiological characterization of the neural responses suggests that the OT/SC is more than a simple motor system. In addition to its control of pre-motor areas, information from the OT/SC is also transmitted to a wide range of areas in the cortex and basal ganglia via thalamic nuclei (Takada et al., 1985; Robinson and Petersen, 1992; Bischof and Watanabe, 1997; Reches and Gutfreund, 2009; Krauzlis et al., 2013), indicating involvement in non-motor functions as well. The emerging hypothesis is that the evolutionary role of the OT/SC is to sort stimuli based on saliency, select the most salient stimulus, and send this information to the appropriate brain regions to direct orienting movements, attention and autonomic responses (reviewed in Boehnke et al., 2011; Knudsen, 2011; Krauzlis et al., 2013). In the following sections, we will review some evidence supporting this hypothesis.

Many studies suggest that neurons in the OT/SC are not simply sensitive to the intensity of the stimulus inside their receptive field (RF) but rather to the saliency of the position represented by their RF at a given instant (Horwitz and Newsome, 1999; McPeek and Keller, 2002; Pluta et al., 2011). This suggests that OT/SC plays a role in target selection (a target is a stimulus worth responding to). For example, in a study to show the role of SC in target selection for saccade execution, Horwitz and Newsome (1999) trained monkeys to choose one of two visual targets based upon the perceived direction of moving dots in a random-dot motion display around the center of fixation. One of the targets was presented inside the RF of the recorded SC neuron and the other well outside the field. Neural recordings revealed that some neurons in the SC responded to the target in their RF only if the direction of the dots was pointing to the direction of their RF location, thus, selecting the target well before the execution of the saccades. Moreover, in well trained monkeys, some neurons responded even without visual stimuli inside their RF, provided that the motion of the dots at the center is contingent with their RF (Horwitz and Newsome, 2001). One interpretation of these results is that the neurons represent the saliency of the stimulus inside their RFs. The learned association between the motion of the dots at the center and the rewarded target enhances the saliency of the position of the RFs pointed out by the motion and, on the other hand, reduces the saliency of the positions not pointed out by the central display.

Further compelling evidence for the causal role of the SC in stimulus selection was provided by inactivation experiments. When a restricted part of the SC is inactivated, monkeys tend to miss the behaviorally relevant targets if they are positioned in the region represented by the inactivated area. Instead, the monkeys choose distracters in areas whose representation is unaffected by the inactivation (McPeek and Keller, 2004). Lovejoy and Krauzlis (2009) expanded this finding to show that focal inactivation of the SC also disrupts the monkey's ability to select stimuli covertly in the inactivated regions (Krauzlis et al., 2013). Interestingly, in both studies, when the stimulus in the inactivated region was presented alone, the monkeys were able to respond

"fnint-08-00001" — 2014/1/14 — 15:38 — page 3 — #3

and covertly attend to the stimulus (McPeek and Keller, 2004; Krauzlis et al., 2013), thus, focal inactivation of the SC does not create a focal sensory neglect. Only in situations when multiple stimuli are presented are the inactivation effects apparent. Thus, SC inactivation specifically disrupts the ability to select the behaviorally relevant stimulus among other non-relevant stimuli.

Analogous results pointing out the importance of the OT/SC in stimulus selection have been reported in other species as well (Ingle, 1975; Woods and Frost, 1977; Marin et al., 2007; Lai et al., 2011; Pluta et al., 2011). Of particular interest here is a series of recent studies in barn owls addressing mechanisms of competitive stimulus selection in the OT (Mysore et al., 2010, 2011; Mysore and Knudsen, 2011b). A stimulus presented alone commonly induces responses in the OT/SC that are larger compared to when it is presented together with other stimuli. This phenomenon, which has been attributed to lateral inhibition, may be interpreted as promoting competition between stimuli (McPeek and Keller, 2002; Knudsen, 2011). Interestingly, Mysore et al. (2010, 2011) have shown some novel properties of this lateral competitive interaction. First, the strength of the suppression does not depend on the distance between the stimuli, which shows that it is a global phenomenon covering the whole visual field (Mysore et al., 2010). Second, many of the neurons are suppressed by competing stimuli only if the strength of the stimulus inside their RF is weaker from the other stimuli but are not suppressed if the RF contains the strongest stimulus in the scene (Mysore et al., 2011). Thus, it seems that the OT tends to code the strength of a stimulus relative to its competitors, a feature that may promote a winner-take-all computation (Mysore and Knudsen, 2011a). Moreover, the stimulus representation in the OT is modulated by top-down connections from forebrain areas (Winkowski and Knudsen, 2006, 2007), thus, possibly allowing for the selection of stimuli not only via their relative strengths but also incorporating cognitive factors such as learned associations, internal states, etc. The mechanism of lateral competitive interactions in the barn owl OT were shown to be mediated by a GABAergic midbrain nucleus (Imc, nucleus isthmi pars magnocellularis) that receives topographic connections from the OT directly and indirectly through a nearby cholinergic nucleus (Mysore and Knudsen, 2013). Thus, the computation of stimulus selection is, at least partly, achieved within the tectal circuitry.

The results described above, as well as numerous other studies, support the above-mentioned hypothesis about the role of the OT/SC as a center of stimulus selection, and shows that this role is preserved across species and across sensory modalities. Two questions, however, remain to be answered. The first question is whether all these elaborate mechanisms of stimulus selection in the OT/SC evolved just for the purpose of gaze and attention. An event that is perceived by an animal as salient typically induces a wide range of behavioral responses (Sokolov, 1963; Barry, 2009; Bradley, 2009). Although a shift in gaze is the most apparent response, a series of autonomic reflexes occur along with it that prepares the body for possible action (Sokolov, 1963; Dean et al., 1989). These include galvanic responses (Bradley, 2009), changes in heart rate (Bradley, 2009), changes in brain wave activity (Naatanen, 1995), and pupillary dilation (Oleson et al., 1972; Stelmack and Siddle, 1982; Bala and Takahashi, 2000). This wide repertoire of responses, which is preserved remarkably across species, has been coined by Ivan Pavlov as the "orienting response" (Sokolov, 1963). Orienting responses can include gaze shifts (overt orienting), but do not have to (covert orienting). In addition, orienting movements can be executed by locomotory muscles of limbs instead of eyes. Therefore, if the hypothesis regarding the role of OT/SC as a center of saliency mapping holds true, then it is predicted that manipulation of activity in the OT/SC will affect orienting responses in general beside eye movements. Evidence supporting this prediction can be found in the literature: microstimulation in the OT/SC can induce pupil dilation responses (PDRs; Netser et al., 2010; Wang et al., 2012); responses in the EMG activity of neck muscles independent of eye or head movements (Corneil et al., 2007); ocular accommodation (Sawa and Ohtsuka, 1994); freezing responses (Dean et al., 1989); increased heart rate (Keay et al., 1988); arousal in cortical EEG (Redgrave and Dean, 1985); and suppression of eye blink reflex (Basso, 1996; Gnadt et al., 1997). Thus, it is evident that apart from its clear role in controlling eye and head movements, the OT/SC is also widely involved in executing a variety of orienting behaviors.

The second question is whether the OT/SC is also involved in temporal saliency detection. As discussed in the introduction, a major part of saliency mapping is history-dependent, that is, a stimulus that is different from its past, is likely to be perceived as salient. However, all of the papers cited above as evidence for the involvement of the OT/SC in saliency mapping, emphasize situations of spatial saliency where the saliency of the target is determined by the difference from the surround. If the OT/SC is indeed a center of saliency mapping in the brain, then it is likely to be involved in the habituation process. Hence, we predict that manipulating tectal activity will disrupt habituation of orienting responses. An indication that indeed this is the case has been provided by Netser et al. (2010). The pupils of barn owls, no different from other species, dilate slightly in response to sudden sounds (**Figure 3A**; Bala and Takahashi, 2000; Spitzer et al., 2003). Netser et al. (2010) measured the pupil diameter of barn owls exposed to a long sequence of identical auditory stimuli. As a result of the long period of repetition, the PDRs became habituated (**Figure 3B**). However, it was shown that if a brief low-level electrical microstimulation is applied to the OT at the site in the map corresponding to the location of the stimulus, the habituated behavioral responses are re-induced (**Figure 3C**). This could not be attributed to general desensitization by the microstimulation as it was significantly less induced by stimulations at other locations in the map (**Figure 3D**). This indicates that the release from habituation was due to manipulations at the local tectal circuitry and therefore supports its involvement in habituation.

Given its suggested role in habituation of the orienting response, we expect to find neural correlates of habituation in the OT/SC. In other words, we expect that neural representation will be strongly modulated by the history of events in a way that suppresses the representation of background stimuli and relatively enhances the representation of odd stimuli. In the following sections, we will discuss the requirements for neural correlates

"fnint-08-00001" — 2014/1/14 — 15:38 — page 4 — #4

**FIGURE 3 | Pupil dilation responses (PDRs) and microstimulation in the OT. (A)** The left image shows a zoomed image of a barn owl's right eye. The image is a video frame taken from an infra-red sensitive video sequence showing the infra-red light reflected from the retina. The arrow points to the edge of the pecten, a retinal landmark absorbing light. The image on the left shows a similar video frame after processing and fitting the pupil edge with an ellipse. The red circle designates the fitted pupil edge. The cross designates the center of the circle and the blue line shows the horizontal edge of the pecten. The diameter of the circle and the average horizontal position of the pecten edge were used to measure pupil dilations and eye movements. **(B)** The graph shows PDRs to three consecutive auditory stimuli given every 12 s. The solid line designates the response to the first stimulus, the dashed line the response to the second stimulus and the dotted line the response to the third stimulus. The base line of all response profiles was reduced to zero level. The horizontal bar indicates the duration of the acoustic stimulus and the vertical line the onset of stimulation. **(C)** Results of coupling acoustic and tectal electrical stimulation. The inset shows the time course of

the stimulation protocol. Auditory stimuli were repeated every 12 s (gray bars). Occasionally, with a probability of 20%, a brief low-level microstimulation was injected shortly before the auditory stimulus (blue bars). The gray curve shows the population average PDR to the repeated auditory stimulus. The red curve shows the population average PDR to the auditory stimuli that followed a microstimulation. Note the release from habituation. **(D)** The inset shows the time course of the experiment. The gray vertical bars indicate auditory stimuli and the blue vertical bars electrical stimulations. Auditory stimuli were repeated every 10 s alternating between two positions: one matching the electrical stimulation site in the tectal map and the other not matching the electrical stimulation site. The gray curves show the population PDRs to the two auditory stimuli. The blue curve shows the population PDR to the non-matched auditory stimuli that were coupled with the microstimulation, and the red curve the population PDR to the matched auditory stimuli coupled with the microstimulation. Microstimulation at the site corresponding with the acoustic stimulus induces a stronger release from habituation. Modified from Netser et al. (2010).

of habituation and then review the literature suggesting such correlates in the OT of the barn owl.

#### **NEURAL CORRELATES OF HABITUATION**

Neural correlates of a particular behavior can be recorded at different levels, ranging from a single neuron to scalp-recorded EEG and fMRI. For example, in scalp-recorded potentials, an auditory component was found that shares some similarities with the phenomenon of habituation. This component is known as mismatch negativity (MMN; Naatanen, 1995). MMN is measured by presenting a sequence of auditory stimuli in which rare sounds or deviants are embedded occasionally. This type of stimulation paradigm is called oddball stimulation (Naatanen, 1995; Ulanovsky et al., 2004; see **Figure 4B**). In such conditions, the evoked potential to the deviant is usually stronger than the evoked potential to the standard. This phenomenon is believed to reflect a habituation-like process whereby the standard stimulus is memorized, allowing the detection and enhancement of responses to deviation from this memory (Nelken, 2004). However, scalp-recorded potentials reflect a global, indirect signal and is therefore limited in its ability to reveal fine details of the neural circuits that compute and represent habituation. For this purpose, identifying neural correlates of habituation at the single-neuron level is preferable.

Habituation is defined as a decline in behavioral response to a sustained or repeatable stimulus that is not fatigue (Thompson, 2009). At the single-neuron level, a reminiscent decline in the neural response to sustained or repeatable stimulus, called adaptation, is a ubiquitous property of sensory neurons (Westerman and Smith, 1984; Gutfreund and Knudsen, 2006). However, it is important to emphasize that habituation is not a mere reduction in

"fnint-08-00001" — 2014/1/14 — 15:38 — page 5 — #5

**FIGURE 4 | Stimulus protocols used to differentiate stimulus-specific adaptation (SSA) from non-specific adaptation. (A)** The priming protocol. In the priming protocol a tuning curve is generated by presenting sequences of stimuli varying randomly over a range of stimuli (f1–fn). This is repeated in two conditions: in one, the stimuli for generating the tuning curve are spaced with a fixed time interval of no stimulation (the control); in the other, in between stimuli, a standard stimulus (fs) is presented several times. In non-specific adaptation (left graph), the tuning curve in the primed condition (gray curve) is expected to be attenuated evenly relative to the control. In SSA, the adaptation is expected to occur stronger near the fs stimulus, resulting in a dip in the tuning curve. **(B)** The oddball protocol. In the oddball protocol, two different stimuli are chosen (f1 and f2) so that both are within the response range of the neuron, giving rise to about the same response. One stimulus is chosen as the standard, being presented repeatedly for a long period. Within this sequence, the second stimulus, called the deviant, is

behavioral response with time. As described above, habituation is a learning process in which a standard or rather unchanging background scene is learned in order to allow an animal to respond selectively to behaviorally relevant stimuli (**Figure 1**; Barry, 2009). Thus, not all types of neural adaptations can serve as neural correlates of habituation. Two major types of adaptation have been described in the literature. One is the non-specific adaptation that depends on the history of activation of the neuron more than on specific features of the stimulus (Calford and Semple, 1995; Brosch and Schreiner, 1997; McAlpine et al., 2000; Ingham and McAlpine, 2004; Furukawa et al., 2005; Wehr and Zador, 2005; Gutfreund and Knudsen, 2006). The other is stimulusspecific adaptation (SSA), an adaptation to a specific stimulus that does not generalize to other stimuli, regardless of how active the neuron was (Ulanovsky et al., 2003, 2004; Antunes et al., 2010). SSA is of particular interest here since, similar to habituation, it depends on the history of the environment rather than on the activity of the neuron. Indeed, the SSA phenomenon has been called single-neuron habituation (Nelken and Ulanovsky, 2007). To distinguish SSA from non-specific adaptation, a variety of stimulation paradigms have been used (**Figure 4**). In all of them a standard stimulus is presented several times in a row to induce adaptation. In SSA the responses to stimuli that are physically close to the standard are expected to be more reduced compared embedded with a relatively low probability. Later, a second block is presented in which the roles are changed: the stimulus that was standard is now deviant, and vice versa. At the end of the experiments, the average responses to the standards are compared with the corresponding responses to the deviants. In non-specific adaptation, we expect no significance differences (left histogram). In SSA, we expect the responses to the deviant to appear consistently above the responses to standards (right histogram). **(C)** The constant order protocol. Two stimuli are presented in a long sequence, starting with several repetitions of f1 then switching to several repetitions of f2, and so on, alternating between the two stimuli. In non-specific adaptation, we expect that the response to the first stimulus in each sub-sequence will be adapted and therefore not larger than the response to the same stimulus when it is last in the sequence. In SSA, we expect a stronger response every time the sequence is switched from one stimulus to the other (right histogram in **C**). Modified from Gutfreund (2012).

to stimuli that are different, whereas in non-specific adaptation the responses to all stimuli are expected to be reduced. In the priming protocol (**Figure 4A**), if SSA exists we expect a dip in the tuning curve near the standard stimulus, whereas if the adaptation generalizes (non-specific) we expect the whole tuning curve to attenuate. In the oddball paradigm (**Figure 4B**), the responses to standard (common) stimuli are compared with responses to the same stimuli when presented rarely (as deviants). SSA implies that the neuron's response to the common stimulus and not to the deviant stimulus is reduced, and, as a result, the response to the deviant stimulus is stronger compared to the same stimulus when presented commonly. In the constant order paradigm (**Figure 4C**), we look at the points of shift between one stimulus type to the other. If SSA exists we expect that the responses to the first stimuli after the shift will be less affected by previous stimuli (because previous stimuli are different), and therefore will be larger compared to subsequent stimuli where previous stimuli are the same. The stimulation protocols illustrated in **Figure 4** may provide an experimental framework for seeking neural correlates of habituation.

Using such protocols, it was shown that SSA is a common phenomenon in the brain and has been observed in visual (Muller et al., 1999), somatosensory (Katz et al., 2006), and auditory (Perez-Gonzalez et al., 2005) pathways. SSA was studied in greater

"fnint-08-00001" — 2014/1/14 — 15:38 — page 6 — #6

detail in the auditory system of cats, rats, and monkeys (Ulanovsky et al., 2003; Perez-Gonzalez et al., 2005; Fishman and Steinschneider, 2012). Neurons sensitive to deviations were found at different levels of the auditory pathway, namely, in the inferior colliculus (IC), the auditory thalamus, and the auditory cortex (Ulanovsky et al., 2004; Anderson et al., 2009; Malmierca et al., 2009; Antunes et al., 2010; Farley et al., 2010; Ayala and Malmierca, 2013; Ayala et al., 2013). SSA is measured in anesthetized as well as awake animals (Richardson et al., 2013) and is therefore pre-attentive. Detailed characterization of SSA in the auditory system revealed that it is highly sensitive to minute deviations from the standard frequency. In the auditory cortex of cats, neurons have been found to respond significantly stronger to stimuli that are deviant from the standard by a frequency difference as small as 0.1% (Ulanovsky et al., 2003). Moreover, it was shown that this adaptation has several time scales ranging from sub-seconds to 10s of seconds (Ulanovsky et al., 2004). This implies that the expected model of the background can be updated on a fast time scale to encompass rapid changes in the stimulus environment, but at the same time, a longer history of stimulation is allowed to affect subsequent responses.

In summary, recent studies on auditory SSA in the cortex, the thalamus and the IC suggest that at the single-neuron level, habituation-like responses are widespread. The origin and mechanisms of this phenomenon are yet to be discovered. And, not less important, the questions of if and how this phenomenon at the single-neuron level is related to habituation at the behavioral level must be answered. Three major problems discussed below hinder the attempts to relate SSA to habituation.

#### **THE MULTIPLE FEATURE PROBLEM**

Stimulus-specific adaptation in the auditory cortex and the IC has been studied mostly using pure tone stimuli where the deviants differed from the standards in terms of sound frequency. Other sensory features such as stimulus intensity, stimulus location, or stimulus length were either not studied or gave rise to poor SSA (Ulanovsky et al., 2003; Farley et al., 2010). An exception is a very recent study in which it was shown that SSA exists in the auditory cortex of rats between stimuli of similar frequencies but differ in their temporal noise structure (Nelken et al., 2013). In nature, a stimulus can differ from what has been in the past along multiple features, i.e., frequency, amplitude, duration, location, etc., or combinations of features. A neuron that is sensitive to changes in the frequency of the stimulus but not to changes in other features is a limited "change detector" that cannot explain the general sensitivity to deviant stimuli observed behaviorally. It is therefore necessary to identify types of SSA that encompass multiple sensory features. Moreover, the phenomenon of habituation is amodal, independent of sensory modality (Thompson and Spencer, 1966). Neural correlates of habituation should therefore not be limited to auditory neurons.

#### **THE MEMORY TRACE PROBLEM**

The memory trace of adaptation is the time it takes from the last stimulus until its effect on the response to the next stimulus wears out. In the laboratory, this is measured by presenting sequences

of stimuli with various inter-stimulus time intervals (ISIs). The minimal ISI in which no adaptation occurs is the duration of the memory trace. SSA in the auditory cortex, the thalamus or the IC has been reported at ISIs as long as 2 s (Ulanovsky et al., 2003; Ayala and Malmierca, 2013) or has only been studied at ISIs < 2 s (von der Behrens et al., 2009; Antunes et al., 2010). Therefore, we can conclude that information about the standard is stored in memory for about 2 s. This poses a major problem because many examples of behavioral habituation have been reported with ISIs of 10s of seconds to minutes, even for short duration stimuli (Thompson and Spencer, 1966; Weinberger et al., 1975; Valentinuzzi and Ferrari, 1997; Bala and Takahashi, 2000; Zimmer, 2006; Dong and Clayton, 2009; Glanzman, 2009). It is therefore necessary to identify a form of SSA that maintains a longer memory trace.

#### **CHANGE DETECTION VERSUS PROBABILITY DETECTION**

Most studies of SSA in the auditory pathways were conducted using standard oddball paradigms whereby a deviant frequency is embedded with a certain probability in the sequence of standard frequency (**Figure 4B**). In such a probabilistic stimulus, SSA implies that the response of the neuron is modulated by the probability of the deviant; the smaller the probability, the larger the response. But sensitivity to probability is not necessarily equal to habituation. Habituation requires learning an expectation rule set by the standard and pointing out any deviations from this rule, while sensitivity to probability simply requires counting the number of stimuli over a period of time and responding accordingly. Rare stimuli are not always salient. For example, a stimulus can appear in a sequence of multiple different stimuli, each being rarely presented, but none is salient compared to the others (Taaseh et al., 2011). The standard oddball paradigm cannot distinguish between the two possibilities. Despite recent attempts to resolve this issue for SSA in the auditory cortex, it still remains an open question (Farley et al., 2010; Taaseh et al., 2011).

In summary, we have provided a short review of the phenomenon of SSA in the auditory system and argued that SSA is the closest reported phenomenon at the single-neuron level to act as a neural correlate of habituation. However, by itself, the wellstudied type of SSA in the main auditory pathway of mammals is not sufficient to account for habituation. We therefore now return to the OT/SC. If, as suggested above, the OT/SC is involved in saliency mapping and habituation, we expect to find a new type of SSA in this structure that is sensitive to multiple stimulus features and modalities and has a longer memory trace.

#### **NEURAL ADAPTATION IN THE OT/SC**

### **NEURAL ADAPTATION IN THE OT/SC IS STIMULUS-SPECIFIC**

Neural adaptation to visual stimuli is robust in the OT, particularly in the deep layers (Woods and Frost, 1977; Boehnke et al., 2011). Habituation-like responses, i.e., SSA, were reported in the OT of pigeons. Some neurons were shown to lose responses completely to repeated visual stimuli, but changes in the type of stimuli resulted in a return of the response to the initial level (Woods and Frost, 1977). Adaptation to visual stimuli in the SC was also reported in the monkey (Fecteau and Munoz, 2005). Recently, adaptation to auditory stimuli was characterized systematically in the barn

"fnint-08-00001" — 2014/1/14 — 15:38 — page 7 — #7

owl's OT (Reches and Gutfreund, 2009; Netser et al., 2011). Oddball and constant order protocols (see **Figures 4B,C**, respectively) were used. It was found that most neurons in the OT responded more strongly to a low probability stimulus than to a high probability stimulus (in the oddball protocol) or more strongly to the first stimulus, which is different from its past compared to the last stimulus (in the constant order protocol). For example, neurons in the OT of the owl are tuned to the interaural time difference (ITD) of the sound, which is the major localization cue for the horizontal position. By presenting sounds through ear phones, it is relatively easy to manipulate the ITD of the sound delivered to the animal (Moiseff and Konishi, 1981). **Figure 5A** shows the average results from constant order experiments in which the stimulus sequence alternated every 10 stimuli between two ITD values (see **Figure 4C**). It can be seen that every time a switch occurred between one ITD to the other, the response increased and adapted again until the next switch occurred. By comparing the response to a stimulus when it is first in its sequence with the response to the same stimulus when it is last in its sequence, it is possible to quantify the SSA. Remarkably, significant changes in neural responses between the first and last stimuli were observed even when the two stimuli differed by an ITD difference as small as 20 μs (Reches and Gutfreund, 2008). Thus, the neural response to an auditory stimulus depends not only on the value of the ITD of

**FIGURE 5 | (A)** The histogram shows the average response of a population of neurons in the OT to a long sequence of stimuli (200 stimuli). The duration of each stimulus was 200 ms and the ISI was 1 s. The sequence alternated between two ITD values; every 10 repetitions the ITD of the sound was switched to the other value (protocol is illustrated in the inset at the bottom of the figure). The graph on the right shows the post-stimulus time histogram (PSTH) of one stimulus when it was first in its sub-sequence (deviant, light red curve) compared to the PSTH for the same stimulus when it was last in its sub-sequence (standard, black curve). **(B)** Same as **A**, but in this case, the two stimuli were narrowband stimuli (width of 1 kHz) differing in their central frequency. **(C)** Same as **A** and **B**, but in this case, the two stimuli differed in intensity. One stimulus (dark blue bars) was softer compared to the other (light red bars). The average response to deviant stimulus was greater in all

cases. Modified with permission from Reches and Gutfreund (2008). **(D,E)** Two examples of tectal recordings showing bimodal enhancement. The raster plots show results of unit responses to a sequence of five repetitions of a stimulus with an ISI of 800 ms. The time course of stimulation is designated by the horizontal black bars. Stimuli were either five visual stimuli, five auditory stimuli, or five congruent bimodal stimuli (visual and auditory stimuli appearing together from the same location). The solid vertical line designates the onset of the first stimulus in the sequence. The dashed vertical lines designate the onset of subsequent stimuli. In both examples **(D,E)**, stronger responses to bimodal stimuli can be seen in the response to the first stimulus in the sequence compared to the unimodal responses. This bimodal enhancement is missing in the responses to the subsequent stimuli. Modified from Zahar et al. (2009).

"fnint-08-00001" — 2014/1/14 — 15:38 — page 8 — #8

the stimulus but also on how this value differs from previous ITD values.

Sensitivity to changes in the input stream was not limited to changes in the ITD of the sound. Testing for changes in the frequency, intensity, or interaural level difference (ILD) of the sound all gave rise to the same basic result. The neurons readily responded more strongly to changes in the input stream along all stimulus dimensions (Reches and Gutfreund, 2008). In all cases (ITD, ILD, intensity, and frequency), SSA was evident and developed rapidly after one trial (**Figures 5A–C**). This similarity between the different features is especially striking, taking into account that the four features are represented and computed in markedly different ways but all still exhibit the same SSA. ITD and ILD, the two primary binaural localization cues, are processed in parallel in two separate and independent brainstem pathways (Takahashi et al., 1984; Takahashi and Konishi, 1988; Adolphs, 1993; Albeck and Konishi, 1995); frequency separation is maintained in the ascending auditory pathways from the cochlea up to the level of the lateral shell of the IC where information is combined across frequency-specific channels (Euston and Takahashi,2002). Soundlevel information is presumably manifested in the firing rates of neurons in the ascending pathways. Therefore, the fact that all four independent acoustic features showed a qualitatively similar pattern of adaptation suggests that SSA is an important property in the neural representation of the auditory scene in the OT. This property possibly underlies the owl's ability to attend and orient abruptly to novel events. For comparison, similar tests were performed in the IC, the main source of auditory information to the OT. Significant SSA in the IC was found only to the frequency of the sound but not to other stimulus features such as ITD, ILD, and intensity (Reches and Gutfreund, 2008). Thus, robust multifeature deviant detection develops in the OT and not earlier in the pathway.

The robustness of SSA in tectal neurons gives rise to an interesting ambiguity problem, i.e., the inability to discriminate between two conditions. For example, a soft sound usually produces weaker responses in the OT compared to a louder sound, however, the same soft sound can induce stronger neural responses than the loud sound when it is deviant in an environment characterized by loud sounds (Reches and Gutfreund, 2008). Thus, neural responses in the tectum seem unable to code the sensory identity of the incoming sound unambiguously. This, together with the finding that OT/SC neurons are mostly broadly tuned to sensory features such as frequency, amplitude modulations, orientation, direction, and modality (Mize and Murphy, 1976; Zahar et al., 2009) is consistent with the hypothesis that the OT represents the location of the stimulus and how salient it is. The exact identification of the stimulus is not a computational task of OT, presumably carried out in a different pathway.

#### **MULTISENSORY INTEGRATION IN THE OT/SC ENHANCES DEVIANCE DETECTION**

In the laboratory, saliency mapping is usually studied in unimodal settings, however, in nature it is primarily an amodal task. The saliency of an event is determined by a combination of modalities (Stein and Meredith, 1993; Pluta et al., 2011), therefore we expect multisensory integration to take place in the saliency mapping pathways. Indeed, multisensory integration is a hallmark of the OT/SC (Stein and Meredith, 1993). Visual, auditory, and somatosensory inputs converge onto the SC, resulting in multisensory neurons that integrate information between modalities (Meredith and Stein, 1986; Meredith et al., 1987). When a visual and auditory stimuli are presented from the same location and at the same time (congruent bimodal stimulation), many neurons enhance their responses dramatically compared with the responses to the visual or auditory stimulus alone. This phenomenon, known as multisensory enhancement, has been characterized in great detail in the SC of cats and monkeys (Wallace et al., 1996; Stanford et al., 2005). Recently, multisensory enhancement was studied in barn owls using paradigms that allow testing for adaptation as well. In a simple adaptation paradigm where the same stimulus was repeated several times (Zahar et al., 2009), it was found that multisensory enhancement of the first stimulus presented before adaptation was robust and comparable to what has been reported in mammals. However, subsequent stimuli presented after adaptation did not show clear multisensory enhancement (**Figures 5D,E**; Zahar et al., 2009). A similar result was shown using oddball stimuli; the multisensory enhancement was much stronger when the stimuli were deviant in the sequence compared to when the stimuli were common (Reches et al., 2010). Thus, multisensory enhancement in the OT is able to increase deviance detection. The mechanisms of this phenomenon are not clear. However, it supports the idea that multisensory integration is used by the OT to enhance saliency mapping (Pluta et al., 2011) by enhancing SSA in congruent bimodal settings (Reches et al., 2010).

#### **THE MEMORY TRACE OF ADAPTATION IN THE OT IS RELATIVELY LONG**

A common notion in the adaptation of neural responses, backed up by computational models of synaptic suppression (Tsodyks and Markram, 1997), is that the dynamics of neural adaptation complies with stimulus duration (Marom, 2009): a short duration stimulus is expected to induce short-lasting adaptation, and vice versa (Varela et al., 1997; Ulanovsky et al., 2004). This concept is in line with most studies of auditory SSA described above. In these studies, the ISIs varied between 300 ms and 2 s, stimulus durations were in the range of 100–500 ms, and the probability of the deviant was 10–15%. None of the papers cited above reported a memory trace longer than 2 s, which is within the time scale of the stimulus timing. However, as mentioned earlier, this relatively short memory trace constitutes a major problem for linking neural SSA with mechanisms of habituation. Behavioral habituation does not comply with the above-mentioned principle of comparable time scales. For example, in the barn owl, reflexive pupil dilation and eye movements to sequences of relatively short stimuli with ISIs of 10–13 s readily habituated and recovered when the stimulus was switched from one type to the another (Bala and Takahashi, 2000; Spitzer et al., 2003; Netser et al., 2011).

To close the time gap between behavioral habituation and SSA in the OT, it is important to examine the memory trace of the SSA. This was done in the OT of barn owls by presenting sequences of identical sounds and measuring unit responses as a function of the position of the stimulus in the sequence. Stimuli

"fnint-08-00001" — 2014/1/14 — 15:38 — page 9 — #9

with ISIs of 10, 30, and 60 s were tested. Remarkably, at all ISIs tested, a single, short (300 ms), and weak (20 dB above the unit's threshold) stimulus was sufficient to induce a significant reduction in the neural response to the second stimulus in the sequence (Netser et al., 2011). Moreover, just like in habituation, the system not only memorized that there was a stimulus earlier but also what type of stimulus. For example, when presenting three short auditory stimuli with an ISI of 60 s, the first two being identical and the third different, the response to the second stimulus was reduced compared to the first, but the response to the odd third stimulus, 60 s later, was completely recovered (Netser et al., 2011). The finding that the memory trace of specific adaptation in the OT can reach time spans of over a minute suggests that this type of adaptation is a neural correlate of behavioral habituation, further supporting the link between tectal neural circuitry and habituation.

# **FINAL REMARKS**

Habituation is commonly known as a reduction in behavioral responses to repeated stimuli. However, the scope of habituation is broader. A seminal work by Sokolov (1963) conceptualizes habituation as a process of learning and memorizing the background for the selection of incoming stimuli that are odd from the background. In this review, we aimed at emphasizing this sometimes forgotten aspect of habituation: the ability to select incoming stimuli if they do not match the stored representation of the background. We reviewed here evidence that the SC in mammals and the OT in birds are involved in the stimulus selection process. In addition, we reviewed recent results suggesting that the activity of tectal neurons may be correlated with habituation of the orienting response. Thus, combined together, these results point to the tectal/colliculi circuitry as a promising model system for studying habituation mechanisms.

In recent years, the avian OT has emerged as a model for studying the neural mechanisms of saliency mapping in space (stimulus competition) and time (habituation; Marin et al.,2005,2007,2012; Wang et al., 2006; Reches and Gutfreund, 2008; Lai et al., 2011; Mysore and Knudsen, 2011b, 2013; Netser et al., 2011). The advantage is that the midbrain circuitry in birds is highly segregated and experimentally accessible (Knudsen, 2011). The overall findings are strikingly similar to findings in other species, including primates. This similarity stresses the importance of a comparative approach to gain an evolutionary perspective on basic elements of animal behavior such as habituation and attention. Here, we focused on recent studies in barn owls regarding SSA in the OT and its possible link to habituation. Yet, two important questions remain to be answered:

#### **WHAT ARE THE NEURAL MECHANISMS OF SSA IN THE OT?**

The neural mechanisms underlying SSA in the auditory pathways as well as in the OT are unknown. One common model to explain SSA is that different stimuli activate separate paths to the recorded neuron and that basic adaptation mechanisms (synaptic depression or intrinsic cellular mechanisms) act at levels where the activation is separated (Eytan et al., 2003). An intriguing observation in the barn owl's SSA is a complete lack of cross-stimulus adaptation at long ISIs, even though the frequency content of

the two stimuli overlapped substantially (Netser et al., 2011). It is therefore unlikely that this type of SSA is accounted for solely by basic response suppressions at lower, frequency-specific levels. To compute the deviancy of complex broadband sounds, a network is required that compares the neural responses to the current stimulus with previous responses based on an integration of information about frequency and amplitude modulations. Details of such a network are yet to be discovered.

#### **WHAT IS THE RELATIONSHIP BETWEEN SSA AND BEHAVIOR?**

The phenomenon of SSA has been studied mostly in anesthetized animals. It is yet to be shown what effects it has on behavior. As previously mentioned, not all types of SSA are linked necessarily to behavioral habituation and saliency mapping. Some may be related to scene analysis or optimal coding (reviewed in Winkler et al., 2009). One approach for studying the relationships between neural adaptation and behavior would be to record behavioral and neuronal responses simultaneously and examine the trial-by-trial correlations between SSA and behavioral habituation. Another approach would be to inactivate brain areas that contribute to SSA and examine the effects on behavioral habituation and on the animal's ability to respond to changes in the environment. Future experiments on these directions are likely to shed light on the neural basis of habituation.

#### **ACKNOWLEDGMENTS**

This work was supported by grants from the Israeli Science Foundation, a grant from the German-Israel Foundation (GIF), and a grant from the Binational Science Foundation (BSF).

#### **REFERENCES**


"fnint-08-00001" — 2014/1/14 — 15:38 — page 10 — #10


"fnint-08-00001" — 2014/1/14 — 15:38 — page 11 — #11


"fnint-08-00001" — 2014/1/14 — 15:38 — page 12 — #12


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 02 October 2013; paper pending published: 06 November 2013; accepted: 02 January 2014; published online: 16 January 2014.*

*Citation: Dutta A and Gutfreund Y (2014) Saliency mapping in the optic tectum and its relationship to habituation. Front. Integr. Neurosci. 8:1. doi: 10.3389/fnint.2014.00001 This article was submitted to the journal Frontiers in Integrative Neuroscience*

*Copyright © 2014 Dutta and Gutfreund. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

"fnint-08-00001" — 2014/1/14 — 15:38 — page 13 — #13

# Adaptation in the auditory system: an overview

# *David Pérez-González1\* and Manuel S. Malmierca1,2 \**

*<sup>1</sup> Auditory Neurophysiology Laboratory (Lab 1), Institute of Neuroscience of Castilla y León, University of Salamanca, Salamanca, Spain <sup>2</sup> Department of Cell Biology and Pathology, Faculty of Medicine, University of Salamanca, Salamanca, Spain*

#### *Edited by:*

*Susanne Schmid, University of Western Ontario, Canada*

*Reviewed by:*

*Pat Michie, University of Newcastle, Australia Bernhard H. Gaese, Goethe University Frankfurt, Germany*

#### *\*Correspondence:*

*David Pérez-González and Manuel S. Malmierca, Auditory Neurophysiology Laboratory (Lab 1), Institute of Neuroscience of Castilla y León, University of Salamanca, C/ Pintor Fernando Gallego 1, 37007 Salamanca, Spain e-mail: davidpg@usal.es; msm@usal.es*

The early stages of the auditory system need to preserve the timing information of sounds in order to extract the basic features of acoustic stimuli. At the same time, different processes of neuronal adaptation occur at several levels to further process the auditory information. For instance, auditory nerve fiber responses already experience adaptation of their firing rates, a type of response that can be found in many other auditory nuclei and may be useful for emphasizing the onset of the stimuli. However, it is at higher levels in the auditory hierarchy where more sophisticated types of neuronal processing take place. For example, stimulus-specific adaptation, where neurons show adaptation to frequent, repetitive stimuli, but maintain their responsiveness to stimuli with different physical characteristics, thus representing a distinct kind of processing that may play a role in change and deviance detection. In the auditory cortex, adaptation takes more elaborate forms, and contributes to the processing of complex sequences, auditory scene analysis and attention. Here we review the multiple types of adaptation that occur in the auditory system, which are part of the pool of resources that the neurons employ to process the auditory scene, and are critical to a proper understanding of the neuronal mechanisms that govern auditory perception.

**Keywords: auditory nerve, inferior colliculus, auditory cortex, stimulus-specific adaptation, mismatch negativity**

# **INTRODUCTION: THE AUDITORY SYSTEM NEEDS TO PRESERVE THE TIMING OF THE SIGNAL**

The challenging task that the auditory system faces is to process naturally occurring sounds, so that they can be identified, characterized, and localized, in order to be able to respond accordingly and in a timely manner. A complication lies in the nature of sound, which consists of rapid variations of the pressure in an elastic medium, usually air for most mammals. One of the basic features of the components of all sounds is their frequency (or how fast the sound waves change) and the auditory brain must be able to extract it very precisely. The range of frequencies that each animal is sensitive to varies greatly. Humans typically can hear sounds from 20 Hz to 20 kHz. Some animals have good low frequency hearing, similar to humans, like the guinea pig (Heffner et al., 1971), but other animals can hear much higher frequencies. For instance, mice can hear sounds over 80 kHz (Heffner and Masterton, 1980) and some bats up to 120 kHz (Koay et al., 1997). In order to process these very rapid variations of the signal, the auditory system requires fast and reliable responses from its elements. Timing information is also essential for the localization of sounds, since it requires a precise encoding of the time at which sounds arrive at each ear. The detection of the minimum change for sound localization in the horizontal plane in humans requires comparing the arrival time at both ears with a precision of a few microseconds (Hafter et al., 1979; Kollmeier et al., 2008).

The timing of action potentials, conveyed with the precision of microseconds, carries acoustic information in all higher vertebrates. For instance, responses of low-frequency auditory nerve fibers are locked to a particular phase of the stimulus waveform (Kiang et al., 1965; Johnson, 1980; Palmer and Russell, 1986), and thus carry a temporal code for sound frequency. The requirement of a precise and faithful transmission of timing information has given rise to the development of certain cellular specializations. The auditory nerve fibers that innervate the anterior ventral cochlear nucleus in mammals have large, specialized calyceal endings, also known as endbulbs, that surround the soma of the target neuron (for a review, see Ryugo and Parks, 2003). In other cells, the synchronization of their responses is enhanced thanks to the convergence of a few auditory nerve fibers through large endbulbs (Ryugo and Sento, 1991; Joris et al., 1994).

This faithful encoding of auditory information is maintained along the ascending auditory pathway up to the auditory cortex (AC), whose neurons are capable of maintaining millisecond precision in the encoding of auditory stimuli (Kayser et al., 2010). But, while the auditory system is so deeply dependent on timing, there are still many instances where adaptation processes take place. Adaptation, as we will consider in this paper, consists on a decrease of the response of a neuron or population or neurons during stimulation, and may manifest itself in several ways. For the sake of simplicity and descriptive purposes, here we differentiate adaptation from habituation, which is commonly used in reference to perceptual and behavioral phenomena, and is more closely related to learning processes. In this review, we will focus on the multiple forms that neuronal adaptation takes through the auditory system.

#### **ADAPTATION OF THE AUDITORY NERVE FIBERS**

Adaptation in the auditory system occurs as early as in the auditory nerve fibers. As has been classically described in other sensory neurons (Adrian, 1926; Adrian and Zotterman, 1926a,b), auditory nerve fibers (**Figure 1**) in all studied species show adaptation (e.g., Nomoto et al., 1964; Kiang et al., 1965; Feng et al., 1991). It takes the form of a higher instantaneous firing rate when a stimulus is switched on, slowing to a lower steady-state rate after a few tens of milliseconds (e.g., **Figure 1**; Sumner and Palmer, 2012). This particular type of adaptation is also known as spike-frequency adaptation, in which a neuron's response to a steady-state stimulus is not maintained at its initially high rate of spiking but instead declines over time to a lower, adapted rate (**Figure 1**). This is a common feature of many sensory neurons (Hille, 1992). This type of response is the origin of the classic "primary" response of auditory nerve fibers, a well-described example of adaptation in the peripheral auditory pathway (Westerman and Smith, 1984; Yates et al., 1985). It is interesting to note that the adaptation is stronger in high frequency fibers than in low frequency fibers (Sumner and Palmer, 2012), especially since low frequency fibers are the ones that show phase locking. This way, the timing information carried by phase locking fibers is preserved. One possible role for adaptation in the auditory system lies in determining the sensitivity of auditory neurons to the stimulus context. The rapid adaptation in auditory nerve fiber responses (Yates et al., 1985; Westerman and Smith, 1987), and the rapid recovery from adaptation (Yates et al., 1983), suggests that the time course of

adaptation in the peripheral nerve fibers might dominate the time course of adaptation in higher centers, unless it is somehow filtered out by neurons at subsequent stages. Indeed, adaptation in these early stages of the auditory pathway may have important implications in the processing of auditory cues at higher centers. In crickets, Givois and Pollack (2000) found that the receptors ipsilateral to the sound source became more adapted than the contralateral ones, since the sound arrives with higher intensity to the ipsilateral side. The different amounts of adaptation produced an imbalance in the interaural difference in response strength, increasing the difficulty of using the interaural level difference as a cue for sound localization. In that situation, they found that the neuronal response latency was more stable, and thus the interaural latency difference was a more reliable cue for sound source localization.

A phenomenon potentially related to adaptation in the auditory nerve is forward masking. It consists in the elevation in the threshold of a signal caused by the presence of a masker sound preceding it in time, and has been the subject of intense study over a number of decades (Harris and Dallos, 1979). Since the preceding, masking sounds caused an apparent reduction of the neuronal responses, adaptation in the auditory nerve has been proposed as a candidate for the neural site of forward masking (Smith, 1977, 1979). However, some studies suggest that forward

overplotted response (gray) from the number of fibers in parentheses, at different levels above their threshold, as indicated on the top of each panel. The mean values are plotted in black. Note how the initially high response rate decreases rapidly to a steady state, in an example of spike-frequency adaptation. Bin width is 0.5 ms. Reproduced from Sumner and Palmer (2012).

masking is better explained by temporal integration models, rather than adaptation (Oxenham, 2001), and so this issue is still a matter of debate.

# **ADAPTATION BECOMES MORE DIVERSE ALONG THE AUDITORY HIERARCHY**

Firing rate adaptation has been also found in other brainstem nuclei. For instance, Finlayson and Adam (1997) studied shortterm adaptation in the superior olivary complex, a group of auditory brainstem nuclei that are involved most notably in the extraction of binaural cues for sound source localization. They found that acoustic stimulation results in rapid and prolonged adaptation in monaurally driven excitation and inhibition of these neurons. For neurons where both the ipsilateral and contralateral inputs are equally affected by adaptation, the effect on localization accuracy is very small. On the other hand, they found that in some neurons the adaptation from ipsilateral and contralateral stimulation is unbalanced, which may affect the coding of localization cues. Finlayson and Adam (1997) conclude that this unbalanced adaptation should cause a poor localization performance by these neurons in noisy conditions.

As we examine higher auditory centers, we can find more complex types of adaptation. The inferior colliculus (IC), the mammalian midbrain auditory nucleus, has received quite considerable attention lately. The IC is a mandatory relay for almost all the ascending auditory information en route to the thalamus and cortex. It receives ascending inputs from most of the lower brainstem nuclei and descending inputs from the cortex (Malmierca and Hackett, 2010; Malmierca and Ryugo, 2011). Therefore, there is no doubt that the IC is strategically located and able to combine and process the information extracted by the previous auditory pathways, so it is not surprising to find more developed neuronal responses.

Processes of spike-frequency adaptation have been described in the IC (Ingham and McAlpine, 2004), in neurons sensitive to interaural phase disparities. In these neurons, it was possible to use a binaural stimulation paradigm that allowed separation of the adaptation of the binaural neurons from that happening at lower monaural levels, such as the auditory nerve fibers. This study revealed that these IC neurons had adaptation dynamics that were rather slow, compared with those calculated for the auditory nerve fibers (Yates et al., 1983, 1985; Westerman and Smith, 1987). The different time constants indicate that the adaptation found in the IC is different from that found in the auditory nerve, and moreover, it is not just inherited from the lower levels.

A different type of adaptation found in the IC is the adaptation of the population coding to stimulus statistics. Dean et al. (2005) studied this type of adaptation regarding the processing of sound level. Mammals can hear sounds extending over an immense range of sound levels with remarkable accuracy. How auditory neurons code sound level over such an extensive range is unclear, since firing rates of individual neurons increase with sound level over only a very limited portion of the full range of hearing. Using stimuli whose intensity changed in a probabilistic way, Dean et al. (2005) found that neural responses were rapidly adjusted by adaptation, in a manner that improved

the coding of the most probable sound levels by the neural population.

Neurons in the IC also show stimulus-specific adaptation (SSA, **Figure 2**). These neurons reduce their responses to a stimulus that is presented repeatedly, but when a novel sound is presented, the same neurons are able to overcome the adaptation and response quickly and vigorously (e.g., **Figures 2** and **3**; Pérez-González et al., 2005; Malmierca et al., 2009). An increase in response strength with the presentation of a stimulus change can be explained by a release from adaption, but particularly when measured in single neurons, it indicates that the underlying adaptation processes are stimulus- (or feature-) specific, thus enabling the system to differentiate stimuli not by their absolute dimensions but by their relative attributes across space and time (Moore, 2003). For these and other reasons, SSA has been proposed to play a role in the attention and the detection of auditory deviance, change and novel stimuli. While SSA was first described in the AC (Ulanovsky et al., 2003), the IC is the lowest nucleus where it is present (Lumani and Zhang, 2010; Zhao et al., 2011; Ayala and Malmierca, 2013; Ayala et al., 2013); it has also been found in the auditory thalamus (Anderson et al., 2009; Antunes et al., 2010; Bäuerle et al., 2011; Antunes and Malmierca, 2014; Duque et al., 2014). The different studies have noted that SSA is stronger in the non-lemniscal divisions of the subcortical auditory nuclei. For instance, it is more prominent in the rostral, dorsal and lateral subdivisions of the IC (Duque et al., 2012), and also in the medial division of the geniculate body (Antunes et al., 2010). On the other hand, in the lemniscal regions, like the central nucleus of the IC and the ventral nucleus of the geniculate body, fewer neurons show SSA, and it is weaker. It seems that SSA is generated *de novo* in each level, and it is not clear that SSA generated in one nucleus propagates to the other, either in a bottom-up or a top-bottom fashion (Antunes and Malmierca, 2011, 2014; Anderson and Malmierca, 2013).

While SSA in the IC was originally described from the neuronal responses using extracellular recordings in animals (**Figures 2** and **3**), including local field potentials (von der Behrens et al., 2009; Patel et al., 2012), a correlate has been found measuring ERP in the human auditory brainstem (**Figure 4**) using the frequencyfollowing response (Slabu et al., 2012), showing that the human IC is able to detect a novel acoustic event occurring among a series of repetitive ones. This is supported by the attenuated human brainstem response (see **Figures 4** and **5**) to a stimulus occurring with a low probability compared with that elicited by the same physical stimulus presented with much higher probability. Their findings suggest that the human auditory brainstem is able to encode acoustic regularities in a memory trace and to detect deviant events based on a comparison process between the current auditory input and the recent auditory history. These results are in agreement with previous studies using the frequency-following response, that showed that the human auditory brainstem encodes stimulus statistics over multiple time scales (Chandrasekaran et al., 2009; Skoe and Kraus, 2010a,b). Similar results have also been observed for cortical neurons (Ulanovsky et al., 2004) and human cortical-evoked potentials (Costa-Faidella et al., 2011). These and other studies have shown the presence of different types of adaptation and deviance-related activity over several time ranges of the

**FIGURE 2 | (A)** In the oddball paradigm, a low probability stimulus (f2, red, "deviant") is embedded in a train of high probability stimuli (f1, blue, "standard"). To compensate the responses to the different physical stimuli f1 and f2, a second sequence is presented where the probability of each stimulus is reversed. Examples of the responses of two neurons recorded using this paradigm is shown in **(B)**, an IC neuron showing SSA (adapting, left) and another not showing SSA (non-adapting, right). Here, f1 and f2 are pure tones of different frequencies, and the frequency difference is the same for both neurons. The top and middle panels in **(B)** show the dot raster in response to sequence 1 (top) and sequence 2 (middle), where the blue dots represent spikes in response to the standard and the red dots the response to the deviant. In the adapting neuron, the response to the standard stimulus decays after the first presentations, while the response to the deviant stimulus remains constant, as a typical example of stimulus-specific adaptation. The bottom panels show the PSTH for the responses to the standard and deviant stimuli, combining the spikes for both stimuli at the same probability. The value of the common SSA index (CSI) is shown for each neuron; CSI values close to one indicate strong SSA while values close to zero indicate weak SSA. Adapted from Duque et al. (2012).

auditory event-related potentials and strongly support the idea of a hierarchically organized system devoted to auditory deviance detection (Grimm et al., 2011; Grimm and Escera, 2012).

The mechanism for SSA is still under investigation, and several options have been put to the test. One of the possibilities under consideration is that SSA emerges from the intrinsic characteristics of the cell, such as the membrane properties. Duque et al. (2012) found that the strength of SSA in IC neurons is not constant within their receptive fields; instead it varies systematically in each neuron, being stronger in the high frequency region as well as near the threshold. If the origin of SSA were based in the intrinsic properties of the cell, its strength should be more uniform within the receptive field, so these results contradict this possibility. Another possible mechanism would be based on the effect of synaptic inhibition, but again, it seems to be unlikely. The pharmacological manipulation of GABAA receptors, in the IC (Pérez-González et al., 2012; Pérez-González and Malmierca, 2012) as well as in the auditory thalamus (Duque et al., 2014), has shown that, while not involved in the generation of SSA, the inhibitory inputs could modulate its strength, acting as a gain control mechanism, in some instances similar to the iceberg effect (**Figure 3**). Instead, a likely mechanism for SSA is one based on the differential adaptation of the inputs to the cell showing SSA (May and Tiitinen, 2010). Comparing multiple stimulus presentation paradigms, Taaseh et al. (2011) proposed that in the AC, SSA is mediated by "adaptation channels," that would span the receptive field of the neuron. In this model, SSA would emerge from the differential adaptation of the channels, as determined by the frequency of the stimuli and their separation. However, other complementary explanations may be needed to fully explain the formation of SSA.

#### **ADAPTATION IN THE AUDITORY CORTEX**

Because of its complex organization and connectivity, including the fact that it is receiving the information that has been extensively processed by all the previous nuclei in the pathway, it is not surprising to find the most numerous types of adaptation processes occur in the AC. Ter-Mikaelian et al. (2007) found that the responses to continuous stimuli adapted with faster kinetics in the primary AC than in the IC, indicating that different temporal filters operate in the different nuclei, which may influence the coding of information in each center. However, in some instances, adaptation processes are quite similar to the counterparts previously described in earlier nuclei. For instance, the SSA found in the AC is quite similar to that found in the IC (Malmierca et al., 2009; Duque et al., 2012) and auditory thalamus (Antunes et al., 2010; Duque et al., 2014), at least in its basic appearance [rat: (Lazar and Metherate, 2003; Szymanski et al., 2009; Farley et al., 2010; Taaseh et al., 2011); cat: (Ulanovsky et al., 2003, 2004)]. However, the range of parameters eliciting SSA seems to be unique for each center, probably reflecting the particular processing capabilities of the neurons. For instance, SSA is elicited by faster repetition rates in the IC (Malmierca et al., 2009) and the thalamus (Antunes et al., 2010) than in the cortex (e.g., Taaseh et al., 2011). On the other hand, the ability to produce SSA with slow repetition rates is not a characteristic exclusive to the cortex, since SSA has been demonstrated in the IC with similar or even longer interstimulus intervals (Zhao et al., 2011; Ayala and Malmierca, 2013). It is also likely that the cortical neurons are capable of processing more complex sequences than those in lower nuclei (e.g., **Figure 6**;Yaron et al., 2012), and indeed, some results suggest that the processing of sequences is hierarchically structured, with higher centers able to process more complex sequences (Althen et al., 2013; Escera and

whether the given stimulus was standard **(A)** or deviant **(B)**. In the baseline condition, the responses to the standard stimulus adapt rapidly after a few presentations **(A**, blue), while the responses to

Malmierca, 2014). However, related studies of subcortical structures are still scarce. Moreover, strong SSA has been reported in the primary AC, being the first lemniscal structure where it has been found, in contrast to the IC and thalamus, where SSA is more prominent in the non-lemniscal subdivisions. The different characteristics of SSA in cortical and subcortical nuclei invites caution when combining the studies performed in each of them. While they probably share some of the mechanisms proposed to create SSA, as explained earlier, each center may add particular conditions that may not extrapolate to the other. For instance, a study demonstrated an analog of SSA in cultured networks of cortical neurons. Eytan et al. (2003) used a paradigm of electrical stimulation similar to the oddball design, and found a depression in the responses to the standard and an increased response to the deviant. Furthermore, this selective enhancement of responses was abolished by blocking GABAergic inhibitory transmission using bicuculline. They proposed that the enhancement of the response to the deviant stimuli was caused by an adaptation of the inhibition, since both standard and deviant stimuli activated the inhibitory circuits. While this is a plausible explanation for cultured cortical neurons, it is unlikely to explain SSA in the IC *in vivo.* We have previously mentioned GABAA-mediated inhibition (**Figure 3**) does not have such effect in the IC (Pérez-González et al., 2012), and hence is another example of the differences in SSA between centers.

However, most examples of adaptation in the AC have been shown by the recordings of evoked potentials, since this part of the brain is very well suited for this technique. For the same reasons, the cortex is the center where most studies have been carried in humans. Using this technique, adaptation is expressed by reduced amplitude of the evoked response to repeated stimulation (Megela and Teyler, 1979). Adaptation in the cortex seems to be involved with the processes of deviance or change detection. These

processes have been studied through experiments that analyze a component of evoked potentials known as mismatch negativity (MMN, **Figure 7**). MMN is evoked by a passive oddball paradigm, where a deviant stimulus is embedded in a train of common, high probability stimuli. MMN is the comparison of the responses to the deviant and common stimuli, resulting in a wave that peaks 150–250 ms after the stimulus onset (Näätänen et al., 1978, 2007). In this context, adaptation would be involved in the reduction of the response to the repetitive, high probability

**FIGURE 4 | Diagrammatic representation of the auditory evoked potential components.** Average from several subjects. The stimuli were clicks presented monaurally, and the EEG was recorded from a electrode in the vertex position and a reference at the mastoid. The components from the first 10 ms (I–VI) correspond to the auditory brainstem responses (ABR), around 20–50 ms correspond to the middle latency responses, and 150–250 ms correspond to the activation of the frontal cortex. Note the logarithmic scales. Redrawn from Picton et al. (1974).

the auditory evoked potentials in response to paradigms of different complexity: the simple deviance detection of the frequency oddball paradigm (right) and a more complex "conjunction" paradigm (left), where the standard stimuli consisted of certain frequency-location combinations and the deviants broke that correspondence, combining one frequency with the opposite location. This figure shows the grand-average for 18 subjects, with the data

range (lower plots). In the middle latency range, there is a reduced response to the standards compared to the deviants, but only in the oddball condition. In contrast, in the long latency range this reduction of the standards occurs for both conditions. These findings suggest a hierarchy in the detection of deviance. Asterisks indicate significant differences. Reproduced from Althen et al. (2013).

stimuli. MMN has been proposed to reflect the comparison of the deviant stimulus with the neuronal trace of the previous stimuli, and it even could be considered some kind of "primitive intelligence" (Näätänen et al., 2001, 2007). One of the characteristics of this change detection system is that it is pre-attentive and automatic, not requiring conscious processing, as indicated by the fact that it persists during sleep and under anesthesia (King et al., 1995; Atienza et al., 2001, 2002). It has been proposed to rely upon a concatenated set of basic adaptation mechanisms and what Bregman referred to as a "bottom-up" or "primitive" grouping (Bregman, 1990; Fritz et al., 2007). The change detection system could be involved in the process of auditory attention (Fritz et al., 2007) or auditory stream segregation (Sussman et al., 2005).

The effects of adaptation in the AC are various. Condon and Weinberger (1991) showed that the repetitive presentation of a stimulus caused long-term frequency-specific changes in the receptive fields of cortical auditory neurons, indicating that adaptation produced a change in the processing of frequency information rather than a general reduction in responsivity. These plastic changes in the AC may be mediated by noradrenergic inputs. The locus coeruleus is a prominent source of noradrenaline which innervates widespread brain regions, including the tectum, the thalamus and the cortex (Sara, 2009). Edeline et al. (2011) used electrical stimulation of the locus coeruleus paired with auditory stimulation to produce plastic changes in the receptive fields of neurons in the AC and thalamus. In fact, it has been shown that the same neurons of the locus coeruleus experience adaptation to auditory stimuli, among others (Herve-Minvielle and Sara, 1995; Vankov et al., 1995). Cholinergic inputs are another

possible candidate for modulating adaptation in the cortex, since its role in processes of cortical plasticity has been shown previously (Metherate and Weinberger, 1989; Kilgard and Merzenich, 1998), but the extent of this possibility awaits future experiments. Acetylcholine is also a tentative modulator of adaptation phenomena in subcortical structures, since it has been shown that cholinergic nuclei in the tegmentum innervate the IC and the auditory thalamus (Motts and Schofield, 2011). Deouell et al. (2007) showed using fMRI that a region in the human medial planum temporale is sensitive to background auditory spatial changes, even when subjects are not engaged in a spatial localization task, and in fact attend the visual modality. During such times, this area responded to rare location shifts, and even more so when spatial variation increased, consistent with spatially selective adaptation.

### **RELEVANCE OF ADAPTATION IN THE AUDITORY SYSTEM**

One of the earliest roles assigned to cortical adaptation is the protection against cortical overstimulation (Megela and Teyler, 1979). This way, the reduction of neuronal activity during repetitive stimulation would have a protective effect, avoiding an overload of the processing systems. As we have mentioned previously, adaptation could also have a role in the detection of auditory change and novelty, as revealed by the experiments on SSA (e.g., Ulanovsky et al., 2003; Malmierca et al., 2009) and MMN (e.g., Escera et al., 1998; Näätänen et al., 2005), as well as auditory attention (Fritz et al., 2007).

Recently, adaptation has been proposed as a way of achieving an efficient coding of the incoming information (Wark et al., 2007). This would suggest that adaptation in stimulus encoding would

be sensitive to the variations in stimulus statistics. By adapting to the current distribution of the stimuli, their values and changes could be represented more precisely. This view is supported by findings like that neurons in the auditory midbrain adjust their responses to the statistics of sound level distributions (Dean et al., 2005), improving the accuracy of the neuronal population code and extending the range of sound levels that can be accurately encoded. This is probably a widespread function for adaptation, since even in grasshoppers it has been found that the recognition of temporal patterns is improved by neuronal adaptation (Ronacher and Hennig, 2004). A number of studies, most notably in the visual system, have suggested a role for adaptation of excitation in scaling neural output to take account of, for example, stimulus variance (Brenner et al., 2000; Fairhall et al., 2001). It is also well described that complex cells of the visual cortex adapt to the local contrast (Ohzawa et al., 1982; Laughlin, 1989; Carandini and Ferster, 1997), the effect being to position a neuron's dynamic range of discharge rates over the relevant range of contrasts.

But not all the response decrements are necessarily related to adaptation. Studying the decrement of the N1 auditory eventrelated potential (**Figure 4**) with stimulus repetition, Budd et al.

(1998) argue that this decrement is based on the separate refractory periods or recovery cycle processes of at least two neural generators contributing to activity in the N1 peak latency range, rather than on an adaptation process. An important feature of the N1 peak of the auditory event-related potential is its systematic reduction in amplitude when the eliciting stimulus is repeated. A major psychophysiological issue regarding the functional nature of N1 amplitude decrement has been the extent to which this response decrement reflects a psychologically relevant process or a more basic neurophysiological process. One method of distinguishing between the distinct processes of adaptation and refractoriness is that amplitude reductions caused by refractoriness should stabilize immediately after repetition of a stimulus while adaptation could entail a more progressive decline in responsiveness (Picton et al., 1976).

### **CONCLUSION**

Adaptation phenomena are widespread in the auditory system, different to habituation, and they appear in multiple forms. Spikefrequency adaptation is already present in the auditory nerve fibers, while nevertheless preserving the timing information. The

Reproduced from Näätänen et al. (2007).

responses of the auditory fibers, despite adaptation are able to carry enough timing information, like the onset and duration of sounds. It is noteworthy to note that phase-locking fibers, which would carry additional timing information, seem to experience weaker adaptation (Sumner and Palmer, 2012). This early balance of adaptation and timing information must be appropriate to allow the processing of basic acoustic features in the brainstem nuclei, such as sound location. It is interesting the fact that other, more elaborate types of adaptation appear in higher levels, maybe because the basic timing information is no longer required. These other types of adaptation could contribute to further processing of the information stream. For instance, in the midbrain the SSA contributes to change and deviance detection. At higher levels, adaptation allows neurons to process more intricate characteristics of the auditory environment, such as abstract relations, complex sequences and regularities, and eventually to contribute to processes like auditory attention and stream segregation.

#### **ACKNOWLEDGMENTS**

We would like to thank Dr. Alan Palmer for his critical and constructive comments on a previous version of the manuscript. Financial support was provided by the Spanish MINECO (BFU2009-07286) and (EUI2009-04083) in the frame of the ERA-NET NEURON to Manuel S. Malmierca, and by a postdoctoral fellowship from the Botín Foundation to David Pérez-González.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 18 October 2013; accepted: 05 February 2014; published online: 21 February 2014.*

*Citation: Pérez-González D and Malmierca MS (2014) Adaptation in the auditory system: an overview. Front. Integr. Neurosci. 8:19. doi: 10.3389/fnint.2014.00019*

*This article was submitted to the journal Frontiers in Integrative Neuroscience. Copyright © 2014 Pérez-González and Malmierca. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited andthatthe original publication inthis journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Stressors impair odor recognition memory via an olfactory bulb-dependent noradrenergic mechanism

# *Laura C. Manella , Samuel Alperin and Christiane Linster\**

*Department of Neurobiology and Behavior, Cornell University, Ithaca, NY, USA*

#### *Edited by:*

*Donald A. Wilson, New York University School of Medicine, USA*

#### *Reviewed by:*

*Susan J. Sara, Collège du France, France Max L. Fletcher, University of Tennessee Health Science Center, USA*

*Carolyn Harley, Memorial University of Newfoundland, Canada*

#### *\*Correspondence:*

*Christiane Linster, Department of Neurobiology and Behavior, Cornell University, W249 Mudd Hall, Ithaca, NY 14850, USA e-mail: cl243@cornell.edu*

Non-associative habituation and odor recognition tasks have been widely used to probe questions of social recognition, odor memory duration, and odor memory specificity. Among others, these paradigms have provided valuable insight into how neuromodulation, and specifically norepinephrine/noradrenaline (NE) influences odor memory. In general, NE levels are modulated by arousal, stress, and behavioral state, but there is sparse evidence of a direct relationship between NE and odor memory in adult rodents. The present study uses simple mild psychological stressors (bright light and sound) to modulate NE levels physiologically in order to probe stressors NE-dependent effect on odor recognition memory. In rats with bilateral bulbar cannulations, we show that these stressors modulate olfactory memory and that this effect is at least partially mediated by the olfactory bulb. Specifically, we show that the presence of stressors during the acquisition of odor memory suppresses memory for an odor when tested 30 min after familiarization to that odor. This suppression is blocked by infusing NE antagonists into the olfactory bulb prior to odor acquisition. Additionally, we find that infusion of bulbar NE is sufficient to suppress odor memory in a manner mimicking that of our stressors. These effects are unlikely to be solely mediated by locomotor/exploratory changes produced by stressors, although these stressors influence certain behaviors not directly related to odor investigation. This study provides important information about how behaviorally relevant changes in NE can influence top-down sensory processing and odor memory.

**Keywords: norepinephrine, noradrenaline, olfactory bulb, odor memory, stress, arousal, locus coeruleus**

# **INTRODUCTION**

Object recognition is a standard behavioral paradigm used to test non-associative recognition memory in animals. An object recognition task generally consists of two trials: a familiarization trial in which the animal has a chance to investigate an object, followed by a test trial in which the animal is presented with the familiar and a novel object. If the animal investigates the novel object more vigorously than the familiar object during the test trial, this is taken as an indication that the animal remembers the familiar object from the previous trial (Bevins and Besheer, 2006). The object recognition task can be used to test memory duration by varying the intertrial interval (ITI) between the familiarization trial and the test trial, as well as memory specificity by varying the similarity of familiar and novel objects. This task relies on the robust neophilic tendency characteristic of rodents and many other animals. This type of task has been used to test a diverse set of questions including, but not limited to the effects of stress (Beck and Luine, 1999; Bowman et al., 2003; Okuda et al., 2004; Scullion et al., 2009; Cazakoff et al., 2010), neuromodulation (Roozendaal et al., 2008; Jurado-Berbel et al., 2010), and aging and gene mutations (Bevins and Besheer, 2006) on memory.

Odor recognition tasks are similar to object recognition tasks in that they also rely on neophilia toward novel versus familiar odorants. Odor recognition and very similar habituation tasks have been successfully used to probe questions of social recognition, odor learning, memory duration, and contributions of various brain areas in these processes (Robert, 1993; Ferreira et al., 1999; Petrulis and Johnston, 1999; Johnston and Peng, 2000; McNamara et al., 2008; Wilson and Linster, 2008; Linster et al., 2009). Habituation has also been used recently using non-social odors to address questions of neuromodulation in general, and its specific effects in the olfactory bulb (OB) (Wilson and Sullivan, 1992; Wilson and Stevenson, 2003; Veyrac et al., 2007; Guerin et al., 2008; Mandairon et al., 2008; Mandairon and Linster, 2009; Escanilla et al., 2010; Freedman et al., 2013).

Non-associative odor learning has been shown to depend critically on a variety of neuromodulators. In particular, manipulation of norepinephrine/noradrenaline (NE) within the OB has pronounced effects on behavior. The OB is the recipient of rich centrifugal projections from noradrenergic neurons in the locus coeruleus (LC) (Aston-Jones et al., 2000; Sara, 2009). To date, most studies that manipulate NE within the OB show an effect on the novelty detection aspect of olfactory non-associative learning but not the memory formation *per se* (but see Guerin et al., 2008). Enhancement of NE in the OB increases spontaneous discrimination of similar odors (Escanilla et al., 2010), whereas blockade of noradrenergic α<sup>1</sup> receptors in the OB impairs spontaneous discrimination of similar odors (Doucette et al., 2007; Mandairon et al., 2008). At the neural level, stimulation of the LC paired with odor presentations in anesthetized animals produced a marked reduction in mitral cell responses (the main output cells of the OB) to odor, and correlated with behavioral recognition memory to the paired odor later in the awake animal (Shea et al., 2008). Therefore, the LC stimulation-induced reduction of mitral cell activity is likely produced by a plasticity that is related to the long term odor recognition memory. A proposed explanation for behavioral recognition via anesthetized habituation in odor responsiveness of mitral cells is related to a study by Chaudhury et al. (2010). In this study, rate of habituation of mitral cell responsiveness to odors in anesthetized animals was comparable to behavioral habituation in awake animals when equating for the total amount of odor exposure and patterns of odor exposure between anesthetized and awake animals. Moreover, brain slice experiments show that NE affects excitability of both mitral and granule cells in the OB (Jiang et al., 1996; Nai et al., 2009; Smith et al., 2009). Computational models of NE modulation in the OB incorporating known physiological features can readily account for the influence of NE on processing of very low odor concentrations and odor discrimination (Escanilla et al., 2010; Linster et al., 2011; Devore and Linster, 2012).

While these studies have shed a great deal of light on the role of NE in OB odor processing, we still know relatively little about the relationship between endogenous patterns of NE release–such as those based on changes in vigilance (Rajkowski et al., 1994), arousal (Aston-Jones et al., 1996, 2000), wakefulness (Pavcovich and Ramirez, 1991; Rajkowski et al., 1994; Sands et al., 2000) or acute stress (Axelrod and Reisine, 1984; Aston-Jones et al., 1996, 2000; Valentino and Van Bockstaele, 2008)—and OB odor processing in adult rodents. However, the roles of stress, glucocorticoids, and NE have been well characterized in neonatal olfactory learning. Critical periods both overlap with and are dependent upon levels of corticosterone, stress, and specifically NE release into the OB (Moriceau and Sullivan, 2004; Moriceau et al., 2009, 2010). In adults, the application of an acute stressor that activates the LC (Valentino and Van Bockstaele, 2008), such as a bright light or sound, or a context that modifies the arousal of the animal can affect memory consolidation and recall, although the precise effects depend on the timing, context, intensity and duration of the stressor involved (reviewed in Joels et al., 2006; Sandi and Pinelo-Nava, 2007). However, these particular effects of acute stress have not been studied in the adult olfactory system.

In the present study, we examine whether natural changes in the NE levels in the OB affect short term odor recognition memory. We use a non-associative odor recognition paradigm to investigate how mild acute stressors—bright light or sound modulate odor memory. Our results demonstrate that the delivery of a mild acute stressor during the familiarization trial of an odor recognition task suppresses odor memory. We further show that this effect is at least partially mediated by OB NE: the effects of stressors can be blocked by bulbar NE antagonists and can be mimicked by infusion of NE into the OB.

# **MATERIALS AND METHODS**

#### **ANIMALS**

Adult male Long Evans Hooded rats, (Charles River Laboratories, Wilmington, MA, USA) initially weighing between 250 and 300 g were used. 12 rats were used for experiment 1 (odor memory duration). A second group of 12 rats was used for experiment 2 (modulation of odor memory). Rats were housed singly in standard laboratory cages (46 × 24 cm). Animals were housed on a reversed 12-h light cycle in constant temperature and food and water available *ad libitum*. All procedures were approved by the Cornell University Institutional Animal Care and Use Committee and were in accordance with NIH guidelines.

#### **CANNULATION SURGERY**

Rats underwent bilateral cannulation surgery under asceptic conditions. Anesthesia was induced using an intraperitoneal injection of ketamine-xylazine (50 and 5 mg/kg, respectively) and maintained using 1–3% isoflurane. At the start of the surgical procedure, rats received subcutaneous injections of prophylactic anitibiotics (Baytril, 5 mg/kg) and analgesics (meloxicam, 2 mg/kg and butorphanol, 2 mg/kg). Rats were then affixed to a stereotaxic instrument (David Kopf Instruments, Tujunga, CA) and the skull overlying the OB s was exposed. Guide cannulae (22 gauge; Plastics One, Roanoke, VA) were implanted bilaterally into the OB s at coordinates with respect to bregma (AP: +8 mm; ML: ±1.5 mm; DV: −4.5 mm), 1 mm dorsal to the target infusion site, and affixed to the skull using skull screws (Plastics One, Roanoke, VA) and dental cement. Each infusion cannula (28-gauge, Plastics One, Roanoke, VA) extended 1 mm from the end of the guide cannula directly into the center of the OB. Dummy cannulae were placed into the guide cannulae to keep the guide cannulae free of debris and prevent infection. Rats were allowed to recover for at least five days before acclimation to the experimental procedures commenced.

### **DRUGS**

All drugs were diluted in 0.9% sterile saline, which was also used as a vehicle control. To block NE signaling in the OB, we used a cocktail of NE antagonists consisting of the α1-receptor antagonist prazosin hydrochloride (1mM, Sigma, Natick, MA), the α2-receptor antagonist yohimbine hydrochloride (2 mM, MP Biomedicals, Solon, OH), and a non-selective β-receptor antagonist alprenolol hydrochloride (120 mM, Sigma, Natick, MA). NE was prepared at a variety of dosages (L-(−)-Norepinephrine (+)-bitartrate salt monohydrate; Sigma, Natick, MA). Drugs were prepared before beginning the experiment, separated into aliquots, and frozen at −20◦C for daily use. During experiments, 6μL of solution was infused into each OB simultaneously using a double-barreled Pump 11 Elite Nanomite Syringe Pump (Harvard Apparatus, Holliston, MA) at a rate of 2μL/min. Drugs were infused 20 min prior to the first trial for a given session. Past studies have shown this volume and rate to be adequate to spread throughout the OB but unlikely to spread beyond the OB (Mandairon et al., 2006).

### **STRESSORS**

We selected bright light and sound stimuli as mild stressors based on their history of being robust modulators of open field behavior (Archer, 1973; Roth and Katz, 1979; Katz et al., 1981) and startle response (Walker and Davis, 1997), as well as promoting phasic NE release from the LC (Koyama et al., 1994). Moreover, these are non-invasive, relatively non-traumatic stressors and pilot studies suggested that animals continue to engage in investigation of odors during familiarization trials regardless of whether a stressor was delivered. For bright light stimulation, a 40 watt desk lamp was placed at the end of the chamber facing inwards in the case of the odor recognition paradigm (**Figure 1**). For sound stress, we used a computer speaker playing music toward the testing chamber (Brick by Boring Brick, performed by Paramore) at a volume, on average, of approximately 70 dB, compared to background level of ambient noise of approximately 56 dB. Both the bright light and sound stressors were present for the familiarization trial (Trial1) for appropriate sessions.

#### **ODORS**

Monomolecular odorants were diluted to approximately 1 Pa vapor partial pressure to normalize the rate of particle dispersion based on concentration and volatility of odors (**Table 1**). For the odor recognition task, we used pairs of perceptually and chemically dissimilar odors. For a given experiment, a rat was presented with all odors sets randomly assigned across experimental conditions (**Table 2**).

#### **ODOR RECOGNITION TASK**

Rats were first acclimated to handling through at least three 10 min daily handling sessions before surgery. After surgery, ratswere

#### **Table 1 | Odor dilutions.**


**FIGURE 1 | Experimental paradigms.** The odor recognition task timeline and paradigm. Infusions were given 20 min prior to the familiarization trial (*Trial 1*) for all sessions (vehicle, NE antagonist cocktail, or a dosage of NE). For sessions where stressors were implemented, the stressor source (lamp for bright light or speaker for sound) was located adjacent to the testing chamber in the position shown in the figure during the entire odor exposure during *Trial 1*. For each trial, two inverted weigh dishes were placed on the top of the lid, in the far back corners. For *Trial 1*, as shown by an overhead view, the familiarization odor, "odor 1," is pipetted onto a one inch square kim wipe into each of two weigh dishes. Between Trials 1

and 2, a variable intertrial interval is implemented (30, 60, 120, 180 min, depending on the particular experiment). For the *Trial 2*, odor 1 is placed in one weigh dish and a novel odor, "odor 2," chemically and perceptually dissimilar from odor 1 is placed in a second weigh dish. The two dishes are placed in a randomized orientation. For each trial, investigation is defined as the rat's nose being within 1 cm of the weigh boat (white box in *Trial 1*, where each arrow is 1 cm). Memory is defined as significantly more investigation of odor 2 compared to odor 1. Additionally, note the quadrant markings at the base of cage in the photo of *Trial 1* that were used for transitions in video scoring.

acclimated to infusion procedures using 0.9% sterile saline infusions and to the testing chamber and room for at least 4 sessions on separate days.

To determine the effects of bright light stress and NE within the OB on odor memory, we used an odor recognition paradigm. This paradigm consists of two three-minute trials, separated by 30, 60, 120, or 180-min intertrial intervals (ITIs). At the start of each trial the rat was transferred from its home cage into the testing chamber: a clean, clear, plastic cage (46 × 24 cm) with a wire lid (**Figure 1**). Following a two-minute acclimation period, odor stimuli were introduced to the rear corners of the cage lid. Odor stimuli were presented by pipetting 60 μl of diluted odors (**Table 1**) onto a one-inch square piece of filter paper (Kim Wipe) inside a weighing dish (**Figure 1**). In *Trial 1* (familiarization trial), both weigh dishes contained the same odor (referred to as the familiar odor) while in *Trial 2* (test trial) one weigh dish was scented with familiar odor and the other was scented with a novel odor (**Table 2**). The orientation of the two odor stimuli during the test trial was randomized (**Figure 1**). The amount of time spent actively investigating each odor—defined as the nose within 1 cm of the weighing dish was determined. High resolution video was recorded during all test sessions and locomotor activity of the rat was determined off-line.

#### **EXPERIMENT 1: ODOR MEMORY DURATION**

To determine how long rats could remember a discrete odor stimulus for use in subsequent experiments, each rat ran through a variety of ITIs between familiarization and test trials. Each rat ran through a randomized block design, experiencing one ITI per day of each of the following ITIs in a randomized order: 30, 60, 120, or 180 min. Four odor sets made up of 4 perceptually and chemically disparate odors (**Table 2**) were used, randomized across sessions so that each odor set had equal representation across each ITI. To be able to compare with the results from experiment 2, rats were


infused with vehicle 0.9% sterile saline into their OBs bilaterally 20 min before the first trial (injection volume 6μl, rate of 2μl/min).

#### **EXPERIMENT 2: EFFECT OF STRESSORS AND BULBAR NE ON ODOR MEMORY**

Next, we determined if short-term odor recognition memory is affected by bright light or sound stressors, and NE blockers (Experiment 2a) or bulbar NE (Experiment 2b). To determine if stressors or bulbar NE infusions enhance odor memory duration, we tested ITIs of 60 and 180 min—ITIs in which our non-stressed rats could not remember, and subsequently an ITI of 30 min—an ITI in which non-stressed rats could remember (**Figure 3**). In order to minimize acclimation to stressors, we interleaved experimental sessions for the two experiments, randomizing all conditions across days (**Table 3**). Additionally, each session was spaced at least three days apart to additionally reduce acclimation to the stressors as well as to the experimental paradigm itself. For Experiment 2a and then Experiment 2b, odor sets from **Table 2** were used, and odor sets were randomized across conditions, similarly to in Experiment 1.

#### *Experiment 2a: does stress enhance odor memory (60 and 180 min ITIs)?*

To determine whether stressors have an effect on odor memory and whether this effect is dependent upon OB NE, we used a 2 × 3 × 2 design where infusions of drug (saline or NE antagonist cocktail) were paired with stressors (no stress, bright light, or sound) at two ITIs (60, 180 min). As a consequence, each rat experienced 20 sessions, each time with different combinations of stress, drug and ITI (**Table 3**).

#### *Experiment 2b: does bulbar NE enhance odor memory (60 and 180 min ITIs)?*

We also determined the effect of NE infusions on odor memory during the odor recognition task. We used a 5 × 2 design, whereby each rat received each drug (vehicle 0.9% saline, 1, 10, 100 or 1000μM NE) at different ITIs (60, 180 min), but in this case without the addition of stressors.



### *Experiments 2c and 2d: do stress or bulbar NE impair odor memory (30 min ITIs)?*

After collecting the data for 60 and 180 min ITIs, we determined whether stress/NE could suppress odor memory by testing an ITI of 30 min—an ITI for which our control rats could remember. The experimental design was identical to testing the longer ITIs above, with the exception that only a 30 min ITI was tested for all conditions. Thus, each rat experienced ten sessions in a randomized order (**Table 3**).

# **ANALYSIS**

All data analyses were done using SPSS statistical software (SPSS, Chicago, IL).

### *Odor memory*

To test for odor memory duration (**Experiment 1**) and effects of NE and stressors on odor memory (**Experiment 2**), we analyzed the amount of investigation in response to familiar and novel odors during the test trial. Wilks' lambda, a multivariate analysis of variance (MANOVA) statistic, was used because MANOVA approaches to repeated-measures analyses of variance do not assume sphericity. Investigation times in response to familiar and novel odor (in seconds) were used as a within subjects factor for **Experiments 1** and **2**. For **Experiment 1**, ITI (30, 60, 120, or 180 min) was set as the between subjects factor. Results from **Experiments 2a and 2c** were analyzed using MANOVA (as above), but with stress (no stress, bright light or sound) and drug (saline or NE antagonist cocktail) as additional between subjects factors. For **Experiment 2**, analyses were run separately for each ITI tested (30, 60, and 180 min). **Experiments 2b and 2d** (NE infusions) were analyzed using MANOVA with drug dosage (saline, 1, 10, 100, or 1000μM NE) as a between subjects factor. For each experiment (**Experiments 1, 2a-d**), *posthoc* tests (Fisher LSD, with α = 0*.*05) determined if rats investigated the novel odor significantly more than the familiar odor during the test trial within each experimental condition.

To further analyze **Experiment 2**, and compare the relative investigation of the novel odor across experimental conditions, we used an ANOVA with experimental group as the between subjects variable and relative investigation times as the dependent variable, followed by pairwise comparisons between experimental groups (Fisher LSD).

Data points containing outliers (more than two standard deviations from mean) were excluded from the data analysis (*<*10% of total). In our experiments these outliers are due to external startling stimuli or distractions.

### *Analysis of investigation time and locomotor activity during the familiarization trial*

To test if differences in odor memory in **Experiment 2** could be due to variability in familiar odor investigation time during *Trial 1* (the familiarization trial), we ran an ANOVA with drug (saline, NE antagonist cocktail) or stress (no stress, light, or sound) as the between subject effects on total time spent investigating the familiarization odor as well as an ANOVA with NE dosage as the between subject effect (1, 10, 100, 1000μM NE), both including data from all ITIs.

To test the effect of drug and stressor on investigation and locomotor activity during the odor recognition task, rats' behavior during the familiarization trials (*Trial 1*) was scored blindly by a trained observer using LabVIEW custom software. The number of rearing bouts per minute (forepaws raised from the cage floor) not including bouts when exploring the weigh boats, proportion of time spent grooming (licking the body, feet and genitals, stroking the face and whiskers with forepaws), and the rate of transitioning from quadrant to quadrants of the testing chamber (**Figure 1**), were measured. Data were pooled across ITIs. An ANOVA was run for each variable (rearing, grooming, and transitioning) defining drug (saline, NE antagonists) and stress (no stress, bright light, sound) as between subjects factors. To determine the effect of all drugs, an ANOVA was run for each variable, defining drug (saline, NE antagonist cocktail, 1, 10, 100, 1000μM NE) as a between subjects factor.

#### **HISTOLOGICAL VERIFICATION OF CANNULA PLACEMENT**

At the end of each experiment, rats were infused with 1% methylene blue solution (in 0.9% sterile saline, 6μl at 2μl/min infusion rate) in order to assess the extent of diffusion within the OB of a single infusion by the beginning of a behavioral trial (Mandairon et al., 2006). After 20 min, animals were perfused transcardially using 0.9% saline followed by 10% neutral buffered formalin. Immediately following brain extraction, the OB and brain were examined visually to assess the spread of dye. Brains were stored in cryoprotectant for at least three days and then sectioned at 40μm and stained with cresyl violet to further determine precise cannula placement within the OB (**Figure 2**). To view and image the slices, we used a Zeiss Stemi 2000C stereo microscope mounted on a transmitted light base with oblique illumination with dual fiber optics for reflected illumination, equipped with a Moticam 2300, 3.0 megapixel color CCD camera (Motic.com) and the Motic acquisition software.

# **RESULTS**

The main goal of this study was to assess effect of mild stressors and intrabulbar infusions of NE on odor recognition memory.

#### **RATS REMEMBER ODOR OBJECTS FOR 30, BUT NOT 60 MIN (EXPERIMENT 1).**

In Experiment 1, we tested the duration of rats' memory using an odor recognition task. **Figure 3** shows the average investigation time of the novel and familiar odor (**Figure 3A**) and the percentage of investigation spent dedicated to the novel odor (**Figure 3B**), both during *Trial 2* and averaged across rats (*n* = 12). These results indicate that rats exhibit a robust novelty response following an ITI of 30 min that is largely diminished by 60 min. A MANOVA revealed a significant effect on investigation time between novel and familiar odorants during the test trial [*F(*1*,* <sup>41</sup>*)* = 20*.*06; *p <* 0*.*001] as well as a significant interaction between investigation time and ITI [*F(*3*,* <sup>41</sup>*)* = 7*.*416; *p <* 0*.*001] indicating that the difference in investigation time was dependent on ITI. *Posthoc* tests (Fisher LSD) show that only rats tested at 30 min ITI investigated the novel odor significantly more than the familiar odor in the second trial (*p <* 0*.*05).

#### **BRIGHT LIGHT, SOUND AND BULBAR NE MODULATE ODOR MEMORY (EXPERIMENT 2)**

Having established the time course of odor recognition memory under control conditions, we examined whether an acute mild stressor or manipulation of bulbar NE during the familiarization trial (*Trial 1*) could modulate odor memory. We tested if memory could be enhanced by these manipulations by using ITIs long enough for control rats to no longer investigate the novel more than the familiar odor (60 and 180 min). We then tested if these manipulations could decrease odor memory duration by using an ITI at which control rats could remember the odor (30 min).

At ITIs longer than 30 min (60 and 180 min), the control group (saline, no stress) do not investigate the novel odor more than the familiar odor (similar to **Experiment 1** in **Figure 3**), and stress and drug do not have an effect on odor memory (**Figure 4**). For **Experiment 2a** (**Figure 4A**), at ITIs of 60 and 180, although MANOVAs reveal a significant overall effect of investigation time between the familiar and novel odor in *Trial 2* when data is pooled across all experimental groups [Wilk's Lambda: 60 min: *F(*1*,* <sup>57</sup>*)* = 10*.*306; *p* = 0*.*002; 180 min: *F(*1*,* <sup>49</sup>*)* = 7*.*442; *p* = 0*.*009] there is no interaction of investigation time with stress [60 min: *F(*2*,* <sup>57</sup>*)* = 2*.*023; *p* = 0*.*142; 180 min: *F(*2*,* <sup>49</sup>*)* = 0*.*649; *p* = 0*.*527] or drug [saline or NE antagonist cocktail; 60min: *F(*1*,* <sup>57</sup>*)* = 0*.*001; *p* = 0*.*974; 180 min: *F(*1*,* <sup>49</sup>*)* = 0*.*168; *p* = 0*.*684]. At these two ITIs, no treatment group investigated the novel odor significantly more than the familiar odor (*p >* 0*.*05 in all cases). For **Experiment 2b** (**Figure 4B**), a MANOVA with level of NE (0, 1, 10, 100, or 1000μM) as a between subjects effect showed no effect of investigation time overall at 60 min [*F(*1*,* <sup>47</sup>*)* = 2*.*162; *p* = 0*.*148], but did show a significant effect of investigation time at 180 min [*F(*1*,* <sup>41</sup>*)* = 16*.*912; *p <* 0*.*001] but

no interaction between drug and investigation time [*F(*4*,* <sup>41</sup>*)* = 0*.*370; *p* = 0*.*823]. No individual treatment group investigated the novel odor significantly more than the familiar odor during *Trial 2* (*p >* 0*.*05 in all cases).

In contrast, **Figure 5** shows that with subsequent testing at a shorter, 30 min ITI, control rats were able to detect the novel odorant indicating memory for the familiar odor (similar to **Figures 3**, **5**). For **Experiment 2c** (**Figure 5A**), with 30 min ITIs, a MANOVA reveals a significant effect of investigation time between familiar and novel odor during *Trial 2* [*F(*1*,* <sup>45</sup>*)* = 11*.*083; *p* = 0*.*002]. However, unlike the longer ITIs, there was a significant interaction with stress [*F(*2*,* <sup>45</sup>*)* = 4*.*765; *p* = 0*.*013] but not drug [saline, NE antagonist cocktail; *F(*1*,* <sup>45</sup>*)* = 2*.*611; *p* = 0*.*113]. Control rats (no stress + saline) displayed significantly higher investigation of novel compared to familiar odors (*p <* 0*.*05), as did rats infused with NE antagonists before stressors were applied (*p <* 0*.*05). Rats stressed with light or sound did not show

**FIGURE 3 | Odor recognition memory duration (Experiment 1). (A)** The graph shows average investigation times for familiar and novel odors during *Trial 2* for 30, 60, 120, and 180 min ITIs. Rats investigate the novel odor more than the familiar odor only at the 30 min ITI, but not at longer ITIs (60, 120, or 180 min ITIs). An ∗ indicates significant values for pairwise comparisons (*p <* 0*.*05) for investigation time between novel and familiar odors (Fisher's LSD tests). **(B)** The graph shows the relative investigation of the novel odor as compared to the familiar odor by showing percentage of total investigation time in which rats investigate the novel odor; this is a measure for odor recognition memory.

**FIGURE 4 | NE and stressors have no effect at longer, 60 and 180 min, ITIs. (A)** Experiments 2a **(A)** and 2b **(B)**: The graphs show the average investigation times to familiar and novel odorants during trials 1&2 at 60 **(A)** or 180 **(B)** min ITI, as a function of stressors

**(Ai)** and **(Bi)** or NE dosage **(Aii)** and **(Bii)**. At these ITIs no treatment groups investigate the novel odor significantly more than the familiar odor during *Trial 2*. Stressors and infusion of NE antagonist cocktail have no effect.

**FIGURE 5 | NE and stressors suppress memory at 30 min ITIs (Experiment 2c and 2d).** These graph show average investigation times to familiar and novel odors in trials 1&2 under different experimental conditions with an ITI 30 min. ∗ indicates a significant difference in investigation time between novel and familiar odor in *Trial 2*. **(A) Experiment 2c**: In non-stressed conditions (either with saline or NE antagonists cocktail infusion into the olfactory bulb prior to the familiarization trial), rats investigate the

significant differences in novel vs. familiar odor investigation. (**Figure 5B**).

For **Experiment 2d** (**Figure 5B**), there was an overall effect of investigation time when NE levels (0, 1, 10, 100, 1000 μM NE)

novel odor significantly more than the familiar odor. However, when either light or sound is implemented during the familiarization trial, rats do not investigate the novel odor more than the familiar odor. The effect of light or sound is counteracted by infusion of NE antagonist cocktail before the familiarization trial. **(B) Experiment 2d**: Saline infused control rats and rats infused with 10μM NE investigate the novel odor significantly more than the familiar odor during *Trial 2*.

were analyzed [*F(*1*,* <sup>41</sup>*)* = 11*.*094, *p* = 0*.*002], as well as significant interaction between investigation time and NE levels [*F(*4*,* <sup>41</sup>*)* = 3*.*2334; *p* = 0*.*042]. Rats infused with 1, 100, and 1000μM NE before the familiarization trial did not investigate the novel odor more than the familiar odor (*p >* 0*.*05), whereas those infused with 10μM NE did (*p <* 0*.*05) (**Figure 5B**).

In summary, the experiments using the 30 min ITI show that memory for the odor, present in saline infused control conditions, is impaired by light or sound stress as well as by NE infusions. Blockade of NE receptors during odor encoding prevents the effects of light and sound stress. The relative investigation of the novel odor as compared to total investigation during the memory trial is commonly used as a measure for the strengths of odor recognition memory (Petrulis and Johnston, 1999; Johnston and Peng, 2000; Veyrac et al., 2007; Guerin et al., 2008). To compare the relative investigation of the novel odor across experimental conditions in the 30 min ITI experiment, we used an ANOVA with experimental group as the main effect and relative investigation time as the dependent variable. Statistical analysis showed a significant effect of treatment group [*F(*9*,* <sup>81</sup>*)* = 3*.*309; *p* = 0*.*002], with light and sound treated saline rats, 1 μM, 100μM and 1000μM NE infused rats investigating significantly different from saline treated non stressed rats (*p <* 0*.*01 in all cases). **Figure 6** shows the summary of relative investigation times across the treatment groups used in experiments 2c and 2d (30 min ITI). This summary clearly shows that (a) light and sound stress during encoding impair memory at 30 min in a manner similar to a range of NE dosages, and (b) that blockade of NE receptors counteract the effect of light and sound, suggesting a noradrenergic contribution in the OB to the effects of light and sound.

#### **CONTROL EXPERIMENTS FOR ACTIVITY PATTERNS DURING TRIAL 1 OF EXPERIMENT 2**

**Figure 7A** shows the results of testing if the effects seen above could be attributed to a change in duration of investigation during the familiarization trial. To do so, we tested the effects of stress (no stress, bright light, sound) and drug (saline, NE antagonist cocktail) on total investigation time during the familiarization trial. ANOVA with stress (no stress, light or sound) or drug (saline or NE blocker) showed no effect of either stress [*F(*2*,* <sup>157</sup>*)* = 1*.*488; *p* = 0*.*220] or drug [*F(*1*,* <sup>157</sup>*)* = 0*.*631]. Additionally, to test if NE dosage affected investigation, we ran an ANOVA with NE dosage (0, 10, 100, 1000μM NE) on investigation time in the familiarization trial [*F(*4*,* <sup>135</sup>*)* = 1*.*848; *p* = 0*.*123]. This suggests results were not due to changes in investigation of the odor during familiarization.

To further investigate how stressors and drugs affect behavior during the familiarization trial, we measured a variety of locomotor/activity measures during the familiarization trial (**Figures 7B–D**). First, we found the rate of rearing during a trial, not including bouts where the animal reared to investigate the odors. ANOVA results show a significant effect of stress on the rate of rearing during the familiarization trial [*F(*2*,* <sup>159</sup>*)* = 6*.*51, *p* = 0*.*0019]. Although a trend exists for both bright light and sound increasing rate of rearing, *posthoc* analysis shows a significant increase as compared to saline infused control rats only with sound (**Figure 7B**). There is no interaction between stress and drug [saline or NE antagonist cocktail; *F(*1*,* <sup>159</sup>*)* = 0*.*44, *p* = 0*.*51]. There was also no effect of any drug treatment (saline, 1, 10, 100, 1000μM NE, NE antagonists) on rearing [*F(*5*,* <sup>150</sup>*)* = 0*.*045, *p* = 1*.*0].

Next, we measured amount of time grooming during the trial, and normalized the value to length of video analyzed (**Figure 7C**). ANOVA results show a significant effect of stress (no stress, light, sound) on percentage of time spent grooming [*F(*2*,* <sup>157</sup>*)* = 6*.*921, *p* = 0*.*001], but no effect of drug (saline or NE antagonist) [*F(*1*,* <sup>157</sup>*)* = 0*.*005, *p* = 0*.*94], or interaction between stress and drug [saline or NE antagonist cocktail; *F(*1*,* <sup>159</sup>*)* = 0*.*124, *p* = 0*.*88]. *Posthocs* (Fisher's LSD) show that bright light and sound both decrease the percentage of time rats spend grooming when compared to no stress control sessions (**Figure 7C**). There was also no effect of NE dosage (saline, 1, 10, 100, 1000μM NE, NE antagonists) on grooming [*F(*5*,* <sup>150</sup>*)* = 0*.*05, *p* = 1*.*0].

Finally, we measured locomotor activity throughout the trial by dividing the chamber into four quadrants and measuring the rate of crossings within these quadrants (quadrants shown in **Figures 1**, **7D**). There was a main effect of stress on rate of transitioning throughout the chamber [*F(*2*,* <sup>157</sup>*)* = 3*.*73, *p* = 0*.*026], whereby *posthoc* LSD tests show that sound but not bright light increase rate of transitions throughout the test chamber when

**FIGURE 7 | Effect of stressors on locomotor activity during the familiarization trial (***Trial 1***).** Stressors enhance exploratory and suppress resting-state activities of rats during the familiarization trial of the odor recognition task while leaving investigation of odor unaltered. **(A)** The graph shows that neither stressor has any effect on total time of odor investigation during the familiarization trial when compared to the non stressed condition. **(B)** The graph shows the number of rearing

time spent grooming. **(D)** The graph shows that sound but not light increases the rate of transitioning between quadrants of the testing chamber compared to the non-stressed condition. ∗indicates significant pairwise comparisons.

compared to no stress controls (**Figure 7D**). However, there was no effect of drug [saline or NE blockers, *F(*2*,* <sup>157</sup>*)* = 0*.*17, *p* = 0*.*90], and no interaction between drug and stress [*F(*2*,* <sup>157</sup>*)* = 1*.*058, *p* = 0*.*35]. Additionally there was no effect of any drug treatment (saline, 1, 10, 100, 1000μM NE, NE antagonists) on rate of transitions [*F(*5*,* <sup>150</sup>*)* = 0*.*87, *p* = 0*.*51].

### **DISCUSSION**

We show here that moderate and acute stressors during acquisition of an odor memory can later suppress that memory. The effect of stressors is impaired by local bulbar blockade of noradrenergic processing and can be mimicked by local infusion of NE (**Figures 5**, **6**). This suggests that NE release from the LC in response to the stressors during memory acquisition (Pavcovich and Ramirez, 1991; Rajkowski et al., 1994; Sands et al., 2000; Valentino and Van Bockstaele, 2008) modulates OB processing to suppress acquisition or expression of the odor memory. Future studies are necessary to determine the particular origin of this effect; whether it abolishes encoding of the memory altogether or alters duration of the odor memory. Regardless of these options, however, these results suggest an importance of state-dependent processing of non-social odors, in this case produced by stressors and dependent upon noradrenergic inputs to the OB.

Our results are in agreement with studies in other brain systems. Chronic stress, for instance, is associated with elevated NE levels and has been linked to suppression of object recognition memory in male rats (Beck and Luine, 1999; Bowman et al., 2003). Additionally, acute stress both during and after acquisition of an object or spatial memory tends to suppress later memory expression (Cazakoff et al., 2010). Although these results are complementary to present findings, it is important to note that odor recognition memory (Cleland and Sethupathy, 2006; Wilson and Sullivan, 2011); involves circuitry and specific plasticity that may not be involved in spatial and object recognition memory (DeVito and Eichenbaum, 2010; Izquierdo et al., 2002), and vice versa. In other cases, NE can enhance memory (Roozendaal et al., 2008), highlighting the notion that NE can produce differing results dependent upon task, dosage, brain area, and timing-dependent effects.

We found that in our behavioral paradigm light and sound stressors alter overall activity levels during the odor acquisition trial when the animal would be stressed (**Figure 7**). Rats were more likely to engage in rearing while being less likely to groom. Additionally, sound but not light enhanced rate of transitions throughout the chamber. Overall, these results suggest that, similar to previous studies (Archer, 1973; Roth and Katz, 1979; Katz et al., 1981), light and sound tend to increase active exploration while inhibiting passive activities such as grooming. These active states have often been shown to relate to higher levels of NE levels in downstream targets of the LC (Pavcovich and Ramirez, 1991; Rajkowski et al., 1994; Sands et al., 2000; Valentino and Van Bockstaele, 2008). Unsurprisingly, the effects we find on exploratory activities are maintained regardless of whether the animal is given bulbar infusions of vehicle or NE antagonists, and are not altered by infusions of any dosage of NE. This maintenance of effect confirms that modulation of other brain areas than the OB promotes these locomotor effects.

With respect to olfactory memory, local blockade of NE receptors in the OB only was sufficient to block the suppression of memory due to stressors; suggesting that the OB at least partially mediates a form of plasticity underlying odor recognition memory (as suggested by the Shea et al., 2008 study). We have previously shown in mice and rats that the formation of a non-associative olfactory memory can be suppressed by bulbar manipulations (Guerin et al., 2008; McNamara et al., 2008; Chaudhury et al., 2010). In mice and rats, bulbar blockade of NMDA receptors impaired the formation of a non-associative memory (McNamara et al., 2008; Chaudhury et al., 2010) and in rats we showed that the associated changes in mitral cell firing were also impaired by local blockade of NMDA receptors (Chaudhury et al., 2010). In mice, lesions of noradrenergic neurons in the LC prevented the formation of odor memory; the effect of these non-specific noradrenergic lesions on olfactory memory formation could be restored by local bulbar infusions of NE (Guerin et al., 2008).

The suppression of memory duration by bulbar NE or stressor is one possible interpretation of our results. Based on data from other groups, alternative interpretations need to be considered. OB NE could act to reinforce the presence of the familiar odor, rendering it more attractive and therefore leading to higher investigation times in a subsequent trial. This type of preference modulation through NE/stressor mechanisms has been shown extensively in neonatal rodents (Moriceau and Sullivan, 2004; Moriceau et al., 2009, 2010). On the other hand, extensive results from our own group using a classical habituation paradigm with repeated odor presentations in the presence of bulbar NE have not shown such a change in odor preference as in these studies habituation itself was not modulated by NE (Wilson and Sullivan, 1992; Wilson and Stevenson, 2003; Veyrac et al., 2007; Guerin et al., 2008; Mandairon et al., 2008; Mandairon and Linster, 2009; Escanilla et al., 2010; Freedman et al., 2013).

Previous data from our group has shown a general enhancement of odor detection and discrimination at very low odor concentration by bulbar NE (Escanilla et al., 2010), which may seem contradictory to the results presented here. This apparent disconnect could potentially be explained by the time course of these behaviors and potentially different roots in the underlying plasticity. The 2010 study showed effects immediately following habituation, while we tested at 30 min and beyond following habituation. In that light, our results may indicate a more interesting tradeoff between an immediate enhancement of detection and discrimination and subsequent suppression of short term memory of that same odor.

Stress, arousal, and NE generally produce an inverted Ushaped curve that predicts performance on memory tasks (Joels et al., 2006; Sandi and Pinelo-Nava, 2007), also seen in recordings of NE modulation in OB slices (Nai et al., 2009) as well as olfactory mediated behaviors (Escanilla et al., 2010). In the present study, the effect of mild stressors could be said to be mimicked by a low dose of NE (1μM NE, **Figures 5**, **6**). The response to NE then follows a non-linear, dose-response curve reminiscent of the typical inverted U-shaped dose response curve of lower and higher dosages suppressing memory performance more so than intermediate dosages (**Figures 5B**, **6**). This result is also similar to the curve describing effects of NE on mitral cell inhibition triggered by NE infusions in OB slices (Nai et al., 2009).

Overall, our results strongly suggest modulation of bulbar processing during memory acquisition by stress-induced NE release into the OB. Using stressors is a valuable, physiologically and behaviorally relevant mechanism to manipulate NE levels behaviorally rather than by infusion or stimulation.

#### **AUTHOR CONTRIBUTIONS**

Laura C. Manella was the primary author, responsible for designing experiments, collecting, recording, analyzing data, and writing the manuscript. Samuel Alperin collected and recorded data. Christiane Linster is the corresponding author and the principle investigator responsible for guiding this research and aiding and advising in analysis of data and writing the manuscript.

#### **ACKNOWLEDGMENTS**

Thank you to the undergraduate research assistants, Peter Bibawi, Michael Dong, Devon McMahan, Ross Markello, Ante Kendes, Sanya Hashmi, and Sarah Lane, who helped collect data for this project. This work was supported by National Institutes of Health Grant 1R01DC009948-01 and National Institutes of Health Cellular & Molecular Training Grant 2T32GM007469.

#### **REFERENCES**


behavioural and pharmacological findings. *Prog. Neuropsychopharmacol. Biol. Psychiatry* 34, 733–741. doi: 10.1016/j.pnpbp.2010.04.002


and reward-motivated discrimination, but not the formation of habituation memory. *Eur. J. Neurosci.* 27, 1210–1219. doi: 10.1111/j.1460-9568. 2008.06101.x


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 11 October 2013; accepted: 05 December 2013; published online: 23 December 2013.*

*Citation: Manella LC, Alperin S and Linster C (2013) Stressors impair odor recognition memory via an olfactory bulb-dependent noradrenergic mechanism. Front. Integr. Neurosci. 7:97. doi: 10.3389/fnint.2013.00097*

*This article was submitted to the journal Frontiers in Integrative Neuroscience.*

*Copyright © 2013 Manella, Alperin and Linster. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Habituation of the responsiveness of mesolimbic and mesocortical dopamine transmission to taste stimuli

# *Maria A. De Luca1,2 \**

*<sup>1</sup> Department of Biomedical Sciences, Neuropsychopharmacology Section, University of Cagliari, Cagliari, Italy <sup>2</sup> National Institute of Neuroscience, University of Cagliari, Cagliari, Italy*

#### *Edited by:*

*Susanne Schmid, University of Western Ontario, Canada*

#### *Reviewed by:*

*Ranier Gutierrez, Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional, Mexico Steven R. Laviolette, University of Western Ontario, Canada James Edgar McCutcheon, University of Leicester, UK*

#### *\*Correspondence:*

*Maria A. De Luca, Department of Biomedical Sciences, Neuropsychopharmacology Section, University of Cagliari, Via Ospedale 72, 09124 Cagliari, Italy e-mail: deluca@unica.it*

The presentation of novel, remarkable, and unpredictable tastes increases dopamine (DA) transmission in different DA terminal areas such as the nucleus accumbens (NAc) shell and core and the medial prefrontal cortex (mPFC), as estimated by *in vivo* microdialysis studies in rats. This effect undergoes adaptive regulation, as there is a decrease in DA responsiveness after a single pre-exposure to the same taste. This phenomenon termed habituation has been described as peculiar to NAc shell but not to NAc core and mPFC DA transmission. On this basis, it has been proposed that mPFC DA codes for generic motivational stimulus value and, together with the NAc core DA, is more consistent with a role in the expression of motivation. Conversely, NAc shell DA is specifically activated by unfamiliar or novel taste stimuli and rewards, and might serve to associate the sensory properties of the rewarding stimulus with its biological effect (Bassareo et al., 2002; Di Chiara et al., 2004). Notably, habituation of the DA response to intraoral sweet or bitter tastes is not associated with a reduction in hedonic or aversive taste reactions, thus indicating that habituation is unrelated to satiety-induced hedonic devaluation and that it is not influenced by DA alteration or depletion. This mini-review describes specific circumstances of disruption of the habituation of NAc shell DA responsiveness (De Luca et al., 2011; Bimpisidis et al., 2013). In particular, we observed an abolishment of NAc shell DA habituation to chocolate (sweet taste) by morphine sensitization and mPFC 6 hydroxy-dopamine hydrochloride (6-OHDA) lesion. Moreover, morphine sensitization was associated with the appearance of the habituation in the mPFC, and with an increased and delayed response of NAc core DA to taste in naive rats, but not in pre-exposed animals. The results here described shed light on the mechanism of the habituation phenomenon of mesolimbic and mesocortical DA transmission, and its putative role as a marker of cortical dysfunction in specific conditions such as addiction.

**Keywords: habituation, dopamine, nucleus accumbens, medial prefrontal cortex, taste stimuli, microdialysis**

# **INTRODUCTION**

Primary motivational states, both positive and negative, are often ruled by the activity of dopamine (DA) neurons in the ventral tegmental area (VTA) and their terminal targets, such as the nucleus accumbens (NAc) and the medial prefrontal cortex (mPFC). In these terminal regions, DA responds to appetitive or aversive stimuli differently depending on specific factors such as stimulus valence, stimulus sensory modality, specific DA neuron subpopulations, different terminal areas studied, and the techniques used for the detection of DA (e.g., microdialysis vs voltammetry; Fibiger and Phillips, 1988; Di Chiara, 1995; Westerink, 1995; Berridge and Robinson, 1998; Schultz, 1998; Redgrave et al., 1999; Di Chiara et al., 2004; Aragona et al., 2009; Lammel et al., 2012; McCutcheon et al., 2012).

The direct correlation between motivational stimulus valence and its effect on the responsiveness of DA transmission has been extensively appreciated by *in vivo* brain microdialysis studies in three different DA terminal areas: NAc shell, NAc core, and mPFC (Bassareo and Di Chiara, 1999; Bassareo et al., 2002). Particularly, it has been observed that the exposure to natural rewards (e.g., highly palatable food) and to salient food taste stimuli (sweet and bitter) increases DA transmission in NAc shell and core and in mPFC of non-food-deprived rats. In NAc shell, but not in NAc core or in mPFC, this response undergoes adaptive regulation after a single pre-exposure to the same taste/food. This response reduces following a recurrent stimulus, and is termed habituation (Thompson and Spencer, 1966; Cohen et al., 1997; Rankin et al., 2009). In NAc shell, habituation to natural rewards is taste specific, and it is reversed by food deprivation of the animals and modified by the presentation of cues associated with the stimulus (Bassareo and Di Chiara, 1999). These observations demonstrate that NAc shell DA is activated by unfamiliar appetitive taste stimuli while DA in the mPFC codes for generic motivational value independently of stimulus valence. Additionally, this underlines the role of NAc shell DA and its habituation in associative learning (Bassareo et al., 2002; Di Chiara et al., 2004).

**Abbreviation:** C, chocolate; DA, dopamine; i.o., intraorally; mPFC, medial prefrontal cortex; NAc, nucleus accumbens; 6-OHDA, 6-hydroxy-dopamine hydrochloride; s.c., subcutaneously; VTA, ventral tegmental area.

In contrast, habituation of DA response is not present after repeated exposure to drugs of abuse (e.g., nicotine, opiates, psychostimulants, cannabinoids), which preferentially stimulate DA transmission in NAc shell as compared to NAc core (Pontieri et al., 1995, 1996; Tanda et al., 1997). However, the use of *in vivo* voltammetry by other labs showed opposite and specific sub-regional changes in DA concentration in response to both cued and unconditioned appetitive stimuli or after cocaine (Aragona et al., 2009; Brown et al., 2011; Badrinarayan et al., 2012).

This review describes experimental evidence for the disruption of habituation of NAc shell DA responsiveness to motivational stimuli *in vivo,* and on the specific circumstances that could contribute to these significant changes. The data here discussed highlight the role of DA in both learning and hedonic processes.

# **SENSITIZATION TO MORPHINE AFFECTS HABITUATION OF MESOLIMBIC AND MESOCORTICAL DOPAMINE RESPONSIVENESS TO TASTE STIMULI**

Morphine administration increases DA transmission in the mesolimbic system, as estimated by *in vivo* brain microdialysis (Di Chiara and Imperato, 1988; Pontieri et al., 1996). Specific experimental protocols of repeated exposure to morphine produced sensitization.

The effect of morphine sensitization on the habituation of the responsiveness of DA transmission to a single pre-exposure to novel, remarkable and unpredictable taste stimuli has been evaluated (De Luca et al., 2011). In order to induce behavioral and biochemical sensitization, a protocol conceived by Cadoni and Di Chiara (1999) has been used. Thus, rats were administered twice a day for three consecutive days with increasing doses of morphine (10, 20, 40 mg/kg s.c.) or saline. After 15 days of withdrawal, rats were administered a precise amount of appetitive sweet chocolate solution through an intraoral cannula (1 ml/5 min, i.o.) during the microdialysis session for NAc shell, core and mPFC dialysate DA analysis.

Our main finding was that opiate sensitization and chocolate pre-exposure exert a differential influence on the response of DA transmission as regards to the specific subdivision of the mesocorticolimbic DA system. **Figure 1** shows the effect of morphine sensitization on the response of NAc shell and core and mPFC DA levels to intraoral sweet chocolate in naive and chocolate pre-exposed rats. We reported that pre-exposure to chocolate produced opposite changes in DA transmission in the mPFC and in the NAc shell (De Luca et al., 2011). In fact, unexpected appearance of habituation in mPFC DA responsiveness to taste stimuli was accompanied by a loss of habituation in NAc shell. Moreover, morphine sensitization was associated with an increased and delayed (50–110 min after chocolate) response of NAc core DA to taste in naive rats while an immediate increase of DA was observed in pre-exposed animals. Similar results were obtained with an aversive stimulus (De Luca et al., 2011). Moreover, although sensitization to morphine is associated with long-term changes in mesolimbic and mesocortical DA responsiveness to taste stimuli, changes in behavioral taste reactivity are lacking. The latter evidence supports the hypothesis that taste-hedonia does not depend on DA (Berridge and Robinson, 1998), thus the increase of DA

**or control rats.** Results are indicated as mean ± SEM of change in DA extracellular levels expressed as the percentage of basal values. Solid symbol, *p* < 0.05 vs basal value; \*, *p* < 0.05 vs Control; *x*, *p* < 0.05 vs shell naive Sensitized; §, *p* < 0.05 vs shell pre-exposed Sensitized. (Adapted from Figures 2 and 3; De Luca et al., 2011).

transmission in these brain regions could arise from the motivational and not from the sensory or hedonic properties of the taste (Bassareo and Di Chiara, 1999; Bassareo et al., 2002).

All of the DA terminal regions studied displayed changes in the habituation (i.e., abolishment vs appearance), which might result in an increased incentive arousal and learning. Notably, the habituation of mPFC DA responsiveness to chocolate releases NAc shell DA from inhibition, thereby abolishing the single-trial habituation of DA. Under this condition, repeated approaches toward a motivational stimulus might be facilitated.

#### **THE ABLATION OF THE mPFC DOPAMINE TERMINALS AFFECTS HABITUATION OF MESOLIMBIC DOPAMINE RESPONSIVENESS TO TASTE STIMULI**

In intact brain, mPFC DA prominently regulates the activity of subcortical DA areas involved in reward and motivation through a complex interaction of many different sub-regions inside the PFC (Murase et al., 1993; Taber and Fibiger, 1995; Kennerley and Walton, 2011). Such control is modulated by DA receptors in the mPFC (Louilot et al., 1989; Jaskiw et al., 1991; Vezina et al., 1991; Lacroix et al., 2000). mPFC DA functions are engaged in cognitive processes (Seamans and Yang, 2004), regulation of emotions (Sullivan, 2004), working memory (Khan and Muly, 2011), and executive functions such as motor planning, inhibitory response control and sustained attention (Fibiger and Phillips, 1988; Granon et al., 2000; Robbins, 2002).

We recently studied the effect of mPFC 6-OHDA lesion on NAc shell and core DA responsiveness to chocolate in naive and chocolate pre-exposed rats. 6-OHDA bilateral infusions in the mPFC modify the responsiveness of NAc DA to gustatory stimuli administered by an intraoral catheter. As shown in **Figure 2**, we observed that in NAc shell of naive subjects the lesion did not change the DA response to intraoral chocolate. However, the lesion of mPFC DA terminals produced an elevated, delayed, and prolonged increase of DA in NAc core in response to an appetitive taste stimulus. In pre-exposed subjects, the lesion did not affect NAc core DA responsiveness to chocolate while it abolished one-trial habituation of NAc shell DA response to sweet taste. After DA terminal lesions, an effect on neither hedonic taste score nor motor activity has been observed (Bimpisidis et al., 2013).

symbol, *p* < 0.05 vs basal value; \*, *p* < 0.05 vs Sham; #, *p* < 0.05 vs Lesioned pre-exposed; §, *p* < 0.05 vs Sham pre-exposed. (Adapted from Figure 6; Bimpisidis et al., 2013).

These observations might suggest that the mPFC DA inhibitory control of DA responsiveness in subcortical striatal areas is different depending on the ventral striatum sub-region studied. Moreover, different sub-regions within the mPFC (e.g., prelimbic, infralimbic) have different projections to different compartments of the NAc. Accordingly, in the NAc shell, which is mostly innervated by the infralimbic area, the cortical-subcortical relationship might work in an opposite manner to that in NAc core.

This is consistent with the different responsiveness of NAc shell and core DA to discrete stimuli and conditions (Di Chiara et al., 2004; Di Chiara and Bassareo, 2007; Aragona et al., 2009; Corbit and Balleine, 2011; Cacciapaglia et al., 2012).

#### **CONCLUSION**

The experimental results here described may help explain, in part, the reason why traumatic PFC injury often facilitates development of drug use disorders (Delmonico et al., 1998). Accordingly, disruption of PFC functions appears following both traumatic conditions (Bechara and Van Der Linden, 2005) and history of drug addiction (Van den Oever et al., 2010; Goldstein and Volkow, 2011). Our data also suggest a correlation between the NAc DA responsiveness to repeated exposure to a motivational stimulus and the control of its activity by the mPFC DA. This refers to mPFC a crucial role in subcortical dysfunction, which may occur in different stages of drug addiction. Similarly, the mPFC plays a crucial role in subcortical dysfunction, which may occur in different stages of drug addiction. Other studies show the direct involvement of mPFC in addiction (Schenk et al., 1991; Weissenborn et al., 1997; Bolla et al., 2003), drug seeking, craving and relapse, which are related to drugs taken either by humans or animals (Kalivas and Volkow, 2005).

Remarkably, we found similarities between the effect of repeated morphine exposure and selective mPFC DA terminal lesions on DA transmission in response to motivational taste stimuli both in NAc shell and in NAc core. However, this correlation seems to exist only after prolonged administration of drugs of abuse, as a single drug exposure did not modify the habituation in NAc shell (De Luca et al., 2012). Moreover, the absence of any relationship between DA habituation and taste reactivity (Berridge, 2000; Bassareo et al., 2002; De Luca et al., 2012) has been validated.

In summary, the specific conditions leading to the abolishment of habituation illustrated in this work clarify the meaning of the habituation phenomenon of mesolimbic and mesocortical DA transmission. Habituation is usually present in NAc shell, but not in NAc core or mPFC, and it is ruled by intact DA transmission within the mPFC. However, the appearance of habituation in the mPFC could be considered as a marker of mPFC dysfunction in its ability to inhibit crucial subcortical functions. This may result in excessive motivation for inappropriate actions originating from a clear loss of impulse control. Finally, yet importantly, NAc DA habituation may be considered *per se* as a marker of drug dependence and its liability.

#### **ACKNOWLEDGMENTS**

This work was supported by grant from the Fondazione Banco di Sardegna, and by RAS LR 7, 2007. The author would like to thank Ms. Tonka Ivanisevic for the help with the preparation of the manuscript.

#### **REFERENCES**


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 14 October 2013; accepted: 13 February 2014; published online: 04 March 2014.*

*Citation: De Luca MA (2014) Habituation of the responsiveness of mesolimbic and mesocortical dopamine transmission to taste stimuli. Front. Integr. Neurosci. 8:21. doi: 10.3389/fnint.2014.00021*

*This article was submitted to the journal Frontiers in Integrative Neuroscience.*

*Copyright © 2014 De Luca. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Habituation of reinforcer effectiveness

# *David R. Lloyd1,2 \*, Douglas J. Medina1, LarryW. Hawk3 ,Whitney D. Fosco3 and Jerry B. Richards 1\**

*<sup>1</sup> Research Institute on Addictions, State University of New York at Buffalo, Buffalo, NY, USA*

*<sup>2</sup> School of Medicine and Biomedical Sciences, State University of New York at Buffalo, Buffalo, NY, USA*

*<sup>3</sup> Department of Psychology, State University of New York at Buffalo, Buffalo, NY, USA*

#### *Edited by:*

*Donald A. Wilson, New York University School of Medicine, USA*

#### *Reviewed by:*

*Antonio Pereira, Federal University of Rio Grande do Norte, Brazil Alfredo Fontanini, Stony Brook University, USA*

#### *\*Correspondence:*

*David R. Lloyd and Jerry B. Richards, Research Institute on Addictions, State University of New York at Buffalo, 1021 Main Street, Buffalo, NY 14203, USA e-mail: drlloyd@buffalo.edu; jrichard@ria.buffalo.edu*

In this paper we propose an integrative model of habituation of reinforcer effectiveness (HRE) that links behavioral- and neural-based explanations of reinforcement. We argue that HRE is a fundamental property of reinforcing stimuli. Most reinforcement models implicitly suggest that the effectiveness of a reinforcer is stable across repeated presentations. In contrast, an HRE approach predicts decreased effectiveness due to repeated presentation. We argue that repeated presentation of reinforcing stimuli decreases their effectiveness and that these decreases are described by the behavioral characteristics of habituation (McSweeney and Murphy, 2009; Rankin et al., 2009).We describe a neural model that postulates a positive association between dopamine neurotransmission and HRE.We present evidence that stimulant drugs, which artificially increase dopamine neurotransmission, disrupt (slow) normally occurring HRE and also provide evidence that stimulant drugs have differential effects on operant responding maintained by reinforcers with rapid vs. slow HRE rates.We hypothesize that abnormal HRE due to genetic and/or environmental factors may underlie some behavioral disorders. For example, recent research indicates that slow-HRE is predictive of obesity. In contrast ADHD may reflect "accelerated-HRE." Consideration of HRE is important for the development of effective reinforcement-based treatments. Finally, we point out that most of the reinforcing stimuli that regulate daily behavior are nonconsumable environmental/social reinforcers which have rapid-HRE. The almost exclusive use of consumable reinforcers with slow-HRE in pre-clinical studies with animals may have caused the importance of HRE to be overlooked. Further study of reinforcing stimuli with rapid-HRE is needed in order to understand how habituation and reinforcement interact and regulate behavior.

**Keywords: ADHD, behavioral regulation, dopamine, drug addiction, obesity, operant conditioning, psychomotor stimulant, sensory reinforcement**

#### **INTRODUCTION**

The central theme of this paper is that an understanding of habituation of reinforcer effectiveness (HRE) provides important insight into the regulation of behavior by reinforcing consequences. This insight is valuable from a theoretical perspective because it provides a more accurate and parsimonious characterization of behavioral phenomena than current theories. Understanding HRE is also valuable because it can be used to improve the understanding and treatment of behavioral dysregulation including: attention-deficit/hyperactivity disorder (ADHD), autism spectrum disorder (ASD), obesity, and drug addiction.

Habituation theory and reinforcement theory make opposing predictions about the effects of reinforcing consequences on behavior. Simply stated, both frameworks describe how an organism responds to the experience of a stimulus. By definition in reinforcement, the repeated presentation of a reinforcing stimulus increases the behavior that produces it, and by definition in habituation, the repeated presentation of a stimulus decreases the behavior observed in response to the stimulus. Therefore, in reinforcement, repeated application of a stimulus is predicted to *increase* a behavior, but in habituation, repeated application of a stimulus is predicted to *decrease* a behavior. While research with animals has consistently indicated that increasing reinforcement frequency increases response rate, a more precise description of the actual relationship between reinforcing stimuli and behavior can be achieved through incorporation of concepts from habituation theory.

Any stimulus capable of serving as a reinforcer has an inherent associated sensory component which may be subject to habituation. Placing food in your mouth, flipping a switch, looking to your left or right, reaching out a hand toward another person, or moving a pen or highlighter across a piece of paper all produce changes in your sensory environment ....sights, sounds, smells, feelings, or tastes. Some of these experiences are novel, some have been previously experienced, some are subjectively good or bad, but all of these sensory components are intrinsically associated with experienced stimuli, and are able to affect the probability of reoccurrence of the response that produced them. It seems likely that these kinds of operant responses and weak reinforcing consequences are typical of the majority of reinforcing contingencies that regulate daily behavior. It has been assumed that the regulation of behavior by weak sensory events with primary reinforcing effects parallels the regulation of behavior by strong consumable reinforcers such as

"fnint-07-00107" — 2014/1/7 — 19:37 — page 1 — #1

food and water in restricted animals. In this paper we suggest that rapid HRE differentiates weak sensory reinforcers from stronger consumable reinforcers.

Research using powerful consumable reinforcers such as food and water has uniformly reported a positive association between response and reinforcement rate. The positive relationship between response and reinforcement rate has been called the matching law (Herrnstein, 1974; de Villiers and Herrnstein, 1976; Heyman, 1983). In contrast, using a weak sensory reinforcer (light-onset), we have reported a negative association between response and reinforcement rate (Lloyd et al., 2012a). We believe that this opposite result was obtained because the effectiveness of light-onset habituates much more rapidly than the reinforcing effectiveness of consumable reinforcers such as food and water. Our interpretation of these results is that the reinforcing effectiveness of light-onset is overpowered by rapid HRE of light-onset.

Sensory reinforcement is understudied. Because most laboratory research with animals uses strong consumable reinforcers such as food and water, sensory reinforcers may be thought of as a "special case." We do not agree. Although weak and transient, sensory reinforcers are important because they are pervasively integrated into our daily experiences. There are an almost infinite number of sensory and social stimuli that may act as sensory reinforcers. We believe that the majority of reinforcers that regulate daily activity are sensory reinforcers. Everything from smiles and praise to flipping a light switch to a car turn signal acts as a source of sensory reinforcement. It may be that the importance of HRE has not been recognized because of the almost exclusive use of powerful consumable reinforcers in pre-clinical studies of reinforcement processes; however, HRE of sensory reinforcers has important implications for both experimental research and applied clinical work with human populations.

#### **DEVELOPMENT OF THE HRE HYPOTHESIS**

There is strong pre-clinical evidence that the effectiveness of reinforcers habituates. Over the past two decades, Frances McSweeney and her students have published a series of reports and reviews about HRE. In early reports, McSweeney (McSweeney, 1992; McSweeney and Hinson, 1992; McSweeney and Roll, 1993; McSweeney and Johnson, 1994; McSweeney et al., 1994a,b, 1995a,b,c,d,e, 1996e; Cannon and McSweeney, 1995; Roll et al., 1995; Weatherly and McSweeney, 1995; Weatherly et al., 1995) observed within-session decreases in operant responding which she later hypothesized may be due to HRE (McSweeney et al., 1996a,b,c,d; Weatherly et al., 1996). Previous to McSweeney's development of the HRE hypothesis, these within-session decreases in responding were commonly explained as being the result of physiological satiation. The research program conducted by McSweeney and her students has provided convincing evidence that within-session changes in operant responding are influenced by habituation and that explanations based on physiological satiation are unlikely. The HRE hypothesis developed by McSweeney and her students has the potential to have a major impact on our understanding of basic reinforcement mechanisms and the application of reinforcement to clinical problems. Objectives of this paper are: (1) to push the HRE perspective beyond consumable reinforcers (e.g., food or water) with consideration of a broader range of sensory reinforcers, (2) to examine the effects of stimulant drugs on HRE, (3) to develop an initial neural/behavioral model of HRE, and (4) to consider the possible clinical impact of HRE.

#### **TEN BEHAVIORAL CHARACTERISTICS OF HABITUATION**

The current understanding of behavioral habituation is well-described by a list of 10 empirical characteristics identified in previous research (McSweeney and Murphy, 2009; Rankin et al., 2009). This list of characteristics is predominately derived from behavioral models where the habituating stimulus occurs *before* a behavior and causes (elicits) the behavior. This differs from operant conditioning where the habituating stimulus occurs *after* a behavior is emitted and is a direct consequence of that behavior. In both cases repeated stimulus presentation leads to a decreased behavioral response, and there is no *a priori* reason to think the habituation characteristics do not describe both behavioral models.

The 10 characteristics are listed in **Table 1** along with a prediction that each characteristic makes when used to describe HRE. All of these predictions are empirically testable, and in some cases are opposite of the prediction of current reinforcement theory. For example, characteristic #4 in **Table 1** predicts that more frequent presentation of a reinforcer will decrease response rate. This contradicts the widely accepted prediction of the matching law (Herrnstein, 1974; de Villiers and Herrnstein, 1976; Heyman, 1983) in which rate of responding is positively related to the rate of reinforcement. As was previously mentioned, we have reported that the rate of responding for a sensory reinforcer (light-onset) can be negatively associated with reinforcement rate (Lloyd et al., 2012a; see discussion in characteristics section).

#### **SENSORY AND BIOLOGICALLY IMPORTANT REINFORCERS**

We initially became interested in HRE while conducting research on sensory reinforcement. We found that responding for sensory reinforcers such as light-onset showed marked within-session declines (Gancarz et al., 2012a; Lloyd et al., 2012a). If we had observed these decreases in responding using a reinforcing stimulus which was consumed (e.g., food or water), we may have attributed the decreases to physiological satiation (i.e., the animals were less hungry or thirsty). However, since light-onset is not consumed, decreases in its reinforcing effectiveness cannot be explained by physiological satiation. In the absence of a satiation mechanism, we turned to McSweeney's HRE hypothesis to explain the observed decreases. As is described later in section "Experimental Analysis of HRE with a Light Reinforcer," we have found a strong correspondence between several predictions listed in **Table 1** and operant responding for light-onset.

Glanzer (1953) was the first to attempt a precise definition of sensory reinforcement-related phenomena. His idea was that exposure to a particular sensory stimulus led to the development of a "quantity of stimulus satiation" specific to that stimulus. With repeated experience of the stimulus, this "quantity" increased and the tendency to respond to the associated stimulus decreased. In their landmark review of habituation, Thompson and Spencer

"fnint-07-00107" — 2014/1/7 — 19:37 — page 2 — #2


**Table 1 | Predictions for habituation of reinforcer effectiveness made by the 10 behavioral characteristics of habituation<sup>1</sup> as described by Rankin et al. (2009).**

<sup>1</sup>*The descriptions of the characteristics of habituation described in this table are abbreviated in order to save space. If clarification is needed please refer to the original descriptions provided by Rankin et al. (2009).*

(1966) point out that Glanzer's stimulus satiation hypothesis can be, "viewed as a formalized restatement of some of the parametric characteristics of habituation." The strong similarity of the characteristics of Glanzer's stimulus satiation hypothesis to the characteristics of habituation described by Thompson and Spencer indicates that habituation has been relevant to even the earliest considerations of sensory reinforcement-related phenomena.

Many researchers have followed Glanzer's example and have referred to what may have been habituation related decreases in responding as satiation. For example decreases in responding for social reinforcers have sometimes been attributed to satiation of a need for social reinforcement. A review by Eisenberger (1970) of studies exploring the extent to which task performance was enhanced by the provision of social praise indicated that individuals who were praised by the experimenter before the start of the task were less responsive to social reinforcers (i.e., had less improvement in performance) than participants who received no praise before the task. An HRE interpretation suggests that the performance decrement is due to a greater number of reinforcer presentations and not satiation of a need for social reinforcement. Unfortunately, the study designs do not allow for clear interpretation of whether satiation or habituation can better account for the data, but this example highlights the relevance of habituation for reinforcers that are highly salient for humans.

"fnint-07-00107" — 2014/1/7 — 19:37 — page 3 — #3

Sensory reinforcers are most often defined as reinforcers that are not biologically important. For example, Kish (1966) defined sensory reinforcement as a "primary reinforcement process resulting from the response-contingent presentation or removal of stimuli of moderate intensity which are not related to some organic need, or removal of aversive stimulation." In another review, Eisenberger (1972) defined sensory reinforcers as "incentives that have no evident tissue-maintenance or reproductive functions." In recent reports, we have referred to sensory reinforcers as"sensory events that do not reduce tissue needs"(Gancarz et al., 2011), as "reinforcers that do not affect homeostatic balance" (Gancarz et al., 2012a), and as "reinforcers that are not biologically important" (Lloyd et al., 2012a). The problem with these types of definitions is that "biological importance or significance" can only be defined very generally and is hard to precisely measure.

With this in mind, we suggest that reinforcers lie along a continuum with the most significant biological reinforcers such as food, water, and painful stimuli at one end of the spectrum and those sensory stimuli, which are often described as "neutral" or "indifferent1," at the opposite end of the spectrum. Stimuli at both ends of the spectrum of biological significance can evoke reflexive responses. For example, food in the mouth elicits salivation, and onset of a light elicits orienting responses. The topographies of these reflexive reactions are markedly different, but perhaps an even more important difference is that orienting responses to the light habituate more rapidly than salivation to food in the mouth of a food restricted animal. When considered as a reinforcing consequence, we expect that the reinforcing effectiveness of light-onset would habituate more rapidly than that of food. We hypothesize that the rate of HRE may distinguish between sensory and biologically important reinforcers and that HRE occurs across the entire continuum.

For consumable reinforcers such as food and water, HRE and physiological satiation provide competing explanations for within-session declines in responding. A water deprived rat may show a within-session decrement in responding because the water previously consumed during the test session alters intra- and/or extracellular fluid levels. It is difficult to completely disregard satiation-based explanations. However, it is possible to support the alternative HRE explanation by using stimulus specificity (characteristic 7) and dishabituation (characteristic 8) tests to rule out satiation-based accounts. For a thorough discussion of various approaches to disentangling satiation- and habituation-based accounts of within-session declines in responding, see McSweeney and Murphy (2009) and Epstein et al. (2009).

In agreement with McSweeney and Murphy (2009), our view is that water and food in the mouth are sensory stimuli that have varying reinforcing effects. Restricting access to water and food is an establishing (motivating) operation that controls the reinforcing effectiveness of the sensory stimuli that are associated with food and water in the mouth (Murphy et al., 2003). In addition to food or water restriction, post-absorptive effects of ingested stimuli may affect the reinforcing effectiveness of predictive sensory stimuli (de Araujo et al., 2012). However, regardless of the level of satiation, or the association of sensory stimuli with post-absorptive factors, it is the contention of this review that the reinforcing effects of sensory stimuli habituate. In the case of sensory reinforcers such as light-onset, satiation or post-absorptive effects do not provide a competing explanation for within-session declines in responding. According to the continuum hypothesis, the effectiveness of all reinforcers (indifferent or biologically important) is strongly regulated by their immediate sensory effects.

# **STIMULANT DRUGS AND DOPAMINE**

Our interest in sensory reinforcement emerged following reports that sensory reinforcers such as light-onset play an important role in the reinforcing effects of the stimulant drug nicotine (Donny et al., 2003a). As is described in the stimulant section below, psychomotor stimulants increase responding for sensory reinforcers, and sensory reinforcers may play an important role in the reinforcing effects of stimulant drugs. We believe that stimulant drug-induced changes in the reinforcing effectiveness of sensory reinforcers are regulated by dopamine (DA). This hypothesis is supported by Redgrave and Gurney (2006) who described a DAbased neural system that controls the reinforcing effects of sensory reinforcers. This theory is described in more detail in section "Neural/Behavioral Model of HRE."

Based on our work with stimulant drugs and sensory reinforcement, we have hypothesized that stimulant drugs disrupt (slow) expression of HRE and that disruption of HRE is an important determinant of stimulant drug effects on operant responding (Lloyd et al., in press). Additionally, the effects of stimulant drugs may depend on the rate of reinforcer habituation, with stimulants having a greater effect on rapidly habituating reinforcers (i.e., sensory-reinforcers such as light-onset) as compared to slowly habituating reinforcers (i.e., biologically important reinforcers such as food and water).

#### **CLINICAL IMPLICATIONS OF HRE**

To date there has been little intentional application of the HRE concept in the clinic. However, many clinical treatment protocols reflect the unintentional use of HRE. An example of unintentional application of HRE in clinical treatment is the common use of reinforcer menus, which allow reinforcing consequences to be varied. From experience, clinicians have learned that varying reinforcers (which prevents HRE) results in greater overall maintenance of the desired behavior (i.e., reinforcer effectiveness). This example of unintentional use of the HRE concept and others are discussed in section "Clinical Significance of HRE."

An example of intentional application of HRE concepts to solve a clinical problem is the innovative research of Leonard Epstein and co-workers who have hypothesized that slow HRE may play an important role in obesity. Epstein's group has repeatedly demonstrated that patterns of within-session responding for food are controlled by stimulus specificity (**Table 1**, characteristic #7). In these demonstrations, responding for one type of food is observed to decrease with repeated food presentation, but can

"fnint-07-00107" — 2014/1/7 — 19:37 — page 4 — #4

<sup>1</sup>In this manuscript we will use "indifferent" instead of the more commonly used "neutral" to refer to sensory stimuli that do not have any biological importance, because "neutral" implies that these stimuli have no effects, which is clearly untrue when referring to them as reinforcers.

be increased by introduction of a new food (Wisniewski et al., 1992; Myers Ernst and Epstein, 2002; Epstein et al., 2003, 2008, 2010; Temple et al., 2007a,b, 2008b). This pattern of intake has also been referred to as sensory-specific satiation, which Epstein et al. (2009) argued is a special case of habituation. These authors note that habituation theory predicts that non-food environmental factors, which are not considered by the sensory-specific satiety concept, influence food intake. For example, Temple et al. (2007a) reported that watching TV while eating causes dishabituation and increases food intake. Evidence that habituation is a determinant of food intake, and that habituation (not satiation) is responsible for within-session declines in responding is extremely important because it implicates a variety of non-foodrelated environmental factors in the problem of obesity. These environmental factors are frequently not considered by current approaches. In further support of the hypothesis that slow-HRE contributes to obesity Epstein's group has reported that overweight children exhibit decreased habituation to food (Temple et al., 2007b; Epstein et al., 2008), and that future increases in body mass index can be predicted by low food habituation rates (HRs; Epstein et al., 2011).

Clinical examples of dysfunctional behavioral regulation (i.e., obesity, ADHD, ASD, and drug addiction) are discussed throughout the manuscript and in more depth in section "Clinical Significance."

# **EXPERIMENTAL ANALYSIS OF HRE WITH A LIGHT REINFORCER**

Following reports that nicotine enhances the reinforcing effectiveness of light-onset (Donny et al., 2003a), we began studying the interaction of the sensory reinforcing effects of a visual stimulus (VS) with the reinforcing effectiveness of other stimulant drugs such as methamphetamine. We used snout-poking as an operant response because rats snout poke at a low rate without any training, allowing us to avoid the need for training the operant response with additional reinforcers such as food or water (cf., bar-pressing).

In our initial (unpublished) attempts to measure lightreinforced behavior, we employed methods that we had previously used to study operant responding for biologically important reinforcers. We placed rats in lighted test chambers and made 5 s of light-*offset* contingent upon snout-poking, according to a fixed ratio (FR1) schedule of reinforcement, with hour-long test sessions conduced 7 days a week to ensure animals had enough time to learn the contingency between responding and light-offset. The results of these initial studies were disappointing. The rate of active snout-poking failed to increase across days, and in fact decreased, and preference for the active snout-poke hole over the inactive hole was inconsistent.

Since this initial attempt to measure light reinforced behavior, we have learned a great deal about light-reinforced responding (Gancarz et al., 2011, 2012a,b; Lloyd et al., 2012a,b). In retrospect, we are able to appreciate some errors of approach we made during our first efforts. We have come to believe that sensory reinforcers are much weaker than consumable reinforcers, in part because the reinforcing effectiveness habituates much more rapidly for light than for consumable biologically important reinforcers. Our understanding of light reinforcement was aided by our gradual rediscovery of the substantial literature on light-reinforced behavior (for reviews see: Lockard,1963; Kish,1966; Berlyne,1969; Tapp, 1969; Eisenberger, 1972). Here we highlight three key aspects of that work.

First, animals should be habituated to the test chamber prior to making light change contingent upon a response. Habituation to the test chamber enhances the effectiveness of light reinforcement (Appel and Hurwitz, 1959; Crowder, 1961; Leaton et al., 1963). Pre-exposure to the test chamber allows the reinforcing effectiveness of competing novel stimuli to habituate; so that when the response-contingent light stimulus eventually occurs it will be novel and/or surprising (and unhabituated) relative to other stimuli in the test chamber.

Second, shorter test sessions should be used in order to observe robust light reinforcement effect. Consistent with an HRE explanation, previous studies consistently show large within-session decrements in responding, with the most responding for the light stimulus occurring during the first minutes of the test session (Roberts et al., 1958; Premack and Collier, 1962; McCall, 1966; Tapp and Simpson, 1966; Gancarz et al., 2011; Lloyd et al., 2012a). Additionally, during hour-long test sessions, habituation may be so extensive that it prevents the spontaneous recovery (**Table 1**, characteristic #2) of responding which otherwise may have been observed in subsequent sessions (**Table 1**, characteristic #6).

Third, and relatedly, longer inter-session intervals should be employed. Increasing the between-session interval attenuates or prevents the decreases in response rate that occur with repeated testing (Forgays and Levin, 1961; Fox, 1962; Premack and Collier, 1962; Eisenberger, 1972). From an HRE perspective, this is spontaneous recovery (**Table 1**, characteristic #6).

The procedures that we initially used to measure the reinforcing effectiveness of light-onset probably would have detected robust effects of consumable reinforcers such as food or water. We believe that this difference is caused by relatively rapid-HRE of sensory reinforcers relative to that of slow-HRE consumable reinforcers like food and water (particularly in food- and water-restricted animals).

Some of the characteristics of habituation are counterintuitive when applied to reinforcement. A fundamental characteristic of habituation is that more frequent stimulation causes more rapid and/or pronounced decrements in responding. If we view the reinforcing effectiveness of light-onset as being a function of habituation, then this indicates that more frequent presentations of the reinforcer will result in a decreased rate of responding. In contrast, a large amount of empirical data from operant experiments using slow-HRE consumable reinforcers like food and water in food and water-restricted animals generally indicates that response rate increases as a function of reinforcer frequency (Herrnstein, 1974; de Villiers and Herrnstein, 1976; Heyman, 1983). This leads to two contradictory predictions regarding the effect of sensory reinforcer presentation rate on responding. An HRE-based hypothesis predicts that less frequent presentation of a reinforcer will decreases HRE and thereby increase response rate. In contrast, previous research with biologically important reinforcers such as food and water indicates that less

"fnint-07-00107" — 2014/1/7 — 19:37 — page 5 — #5

frequent presentation of a reinforcer should decrease response rate.

To investigate this relationship, we examined responding for light-onset presented according to FR1 and VI 6 min schedules of reinforcement (Lloyd et al., 2012a). On the FR 1 schedule, every snout-poke produced the VS (5 s light-onset). On the VI 6 min schedule, snout-poking produced light-onset on the average of every 6 min. The results are depicted in **Figure 1**. As predicted by the HRE hypothesis, higher rates of responding occurred in the VI 6 min condition, the condition in which reinforcement was*less* frequent. This result indicates the importance of HRE as a regulator of reinforcement and that the functional relationship between reinforcer rate and response rate may be different for non-consumable purely sensory reinforcers and consumable biologically important reinforcers. We believe that this difference is because sensory reinforcers habituate more rapidly than consumable reinforcers in deprived animals.

The data depicted in **Figure 1** illustrate a number of important characteristics of habituation described by Rankin et al. (2009) and listed in **Table 1**. The first behavioral characteristic of habituation (**Table 1**) is that, "repeated application of a stimulus results in a progressive decrease in some parameter of a response to an asymptotic level." For the two reinforcement schedules shown in **Figure 1**, responding was greatest during the first 6 min epoch of the test session and then decreased. The FR1 schedule produced reliable within-session decreases in responding for all test sessions. The VI 6 min schedule produced reliable within-session decreases in responding during tests 3–5. For the FR1 schedule, where there are more repetitions of the reinforcer, the pattern of results is consistent with responding being reduced to "asymptotic levels." In addition to the present data, previous studies have consistently shown both between- and within-session decrements in VS reinforced responding (Roberts et al., 1958; Premack and Collier, 1962; McCall, 1966; Tapp and Simpson, 1966; Gancarz et al., 2012a).

The second behavioral characteristic of habituation (**Table 1**) is that, "If the stimulus is withheld after response decrement,

**was contingent on snout-poking.** Light was presented according to one of three schedules (FR 1, VI 1 min, and VI 6 min; data for VI 1 min not shown). The experiment had two phases, one in which animals were pre-exposed to an experimental chamber for 10 sessions, followed by the second phase of 10 sessions in which animals could snout-poke into one of two holes to produce 5 s light-onset. The data are depicted as two session blocks and show the rate of responding in 6 min epochs of each 30 min test session. BL 1 indicates responding at the end of the pre-exposure phase when there was no response-contingent light-onset. Tests 1–5 indicate two session blocks of light-contingent responding.

the response recovers at least partially (spontaneous recovery)." Following the increases in responding due to the initial primary reinforcing effects of the VS, there was clear evidence of spontaneous recovery from the decrements in responding that occurred during the previous test session. That is, responding during the first 6 min epoch of the test session is greater than responding during the last 6 min epoch of the previous test session. In addition, previous studies of the reinforcing effectiveness of visual stimuli have shown that increasing the intersession intervals results in greater recovery of responding (Forgays and Levin, 1961; Fox, 1962; Premack and Collier, 1962; Eisenberger, 1972). These data are consistent with the interpretation that longer intervals between test sessions result in greater spontaneous recovery of the reinforcing effectiveness of visual stimuli.

The third behavioral characteristic of habituation (**Table 1**) is that,"After multiple series of stimulus repetitions and spontaneous recoveries, the response decrement becomes successively more rapid and/or more pronounced." For the FR1 schedule, betweensession decreases in total session responding were observed for all test sessions following test session 1. For the VI 6 min schedule, between-session decreases in total session responding were observed for the four test sessions following test sessions 5 and 6. For both schedules, the within-session pattern of responding generally indicates that the decrease in responding from the first 6 min epoch to the second 6 min epoch became larger with repeated testing. Taken together, the data are consistent with the interpretation that within-session decreases in reinforcer effectiveness due to habituation are accelerated by repeated cycles of testing and recovery.

The fourth behavioral characteristic of habituation (**Table 1**) is that, "More frequent stimulation results in more rapid and/or more pronounced response decrements." Consistent with this characteristic, the schedule providing the most frequent reinforcement (FR1) caused faster and more pronounced decreases in within-session responding than were observed in the schedule providing the least frequent reinforcement (VI6). This inverse relationship is consistent with the interpretation that the initial reinforcing effectiveness of the VS was decreased by the frequency of its occurrence. One interpretation of these data is that the initial primary reinforcing effectiveness of the VS was equivalent and that reinforcer effectiveness was decreased by habituation. The degree of habituation was determined by the schedule of reinforcement, with schedules that permitted higher frequencies of response-contingent VS presentation resulting in more rapid habituation.

The seventh behavioral characteristic of habituation (**Table 1**) is that, "Within the same stimulus modality, the response decrement shows *stimulus specificity.*" For HRE, the test for stimulus specificity is to present a stimulus contingent upon the response that has stimulus properties that are different from the original reinforcing stimulus. If the observed response decrements are caused by habituation and not by adaptation or fatigue, the animal should show increased responding to produce the novel stimulus.

Demonstrations of stimulus specificity are important because they provide a litmus test for determining if declines in responding can be attributed to habituation. Evidence for stimulus specificity

"fnint-07-00107" — 2014/1/7 — 19:37 — page 6 — #6

of response decrements in light reinforced responding is presented in **Figure 2A** (Wang et al., in preparation). In this experiment we first trained rats to respond to produce light-onset that originated from a light located at the front of the test chamber. During a subsequent 1 h challenge test, subjects were able to respond to produce light in the front of the chamber for the first 30 min of the test session, and for the following 30 min of the test session, responding activated a light at the rear (instead of the front) of the test chamber. Robust within-session decreases in responding were observed, indicating HRE. Shifting the light to the rear of the chamber increased responding above BL levels during the last 30 min of the test session, demonstrating that the within-session decrease in responding was dependent upon specific stimulus properties of the light. As stated above, this is important because it shows that within-session decreases in responding were not caused by factors such as motor fatigue or sensory adaptation. Had these factors been responsible for the within-session decrease in responding, the change in location of the light would not have resulted in any increases.

The eighth behavioral characteristic of habituation (**Table 1**) is that, "Presentation of a different stimulus results in an increase of the decremented response to the original stimulus." A test involving restoration of responding by introduction of a noncontingent external stimulus is termed a test of *dishabituation*. Evidence for dishabituation by a non-contingent external stimulus on responding for a sensory reinforcer is presented in **Figure 2B** (Wang et al., in preparation). A loud warbling tone was presented for 6 min during the middle of a 1 h test session (minutes 30–36). (Note that this is different from the stimulus specificity test above because the reinforcer was not changed). **Figure 2B** shows that responding was increased during presentation of the warbling tone. The data in **Figure 2B** are plotted in 10 6-min epochs for the 1 h test sessions. Robust within-session decreases in responses were observed, indicating HRE. Responding during each epoch is plotted as a percent of total responding during the first epoch. Baseline (BL) indicates responding during the two previous sessions. Test-session responding was divided by the number of responses during BL for each epoch. Presentation of the tone increased responding above BL levels during min 30–36 (epoch 6), ruling out motor fatigue or sensory adaptation.

Rankin et al. (2009) placed particular emphasis on stimulus specificity and dishabituation as evidence for habituation because both tests effectively rule out fatigue or sensory adaptation as explanations for decrements in responding. Previous studies have shown that both tests of stimulus specificity (Aoyama and McSweeney, 2001; Epstein et al., 2003; Kenzer et al., 2013) and tests of dishabituation (McSweeney and Roll, 1998; Aoyama and McSweeney, 2001; Kenzer et al., 2013) are able to restore responding for consumable reinforcers (see Epstein et al., 2009, for a review). The work reviewed demonstrates both phenomena also occur with sensory reinforcers in rodents.

Evidence that the reinforcing effectiveness of sensory (nonconsumable) reinforcers also habituates in humans has been documented in a recent paper by Kenzer et al. (2013). This study used tests of stimulus specificity and dishabituation to demonstrate that within-session decreases in response rate were due to

HRE for both food and two types of sensory reinforcers. In this study, participants clicked a square on a computer screen with a mouse. After each click the cursor controlled by the mouse was reset to the bottom corner of the screen. Click responses in the square were reinforced according to FR schedules of reinforcement. Different kinds of reinforcers were tested, including food items, social statements (e.g., "good job") and pictures. When within-session response rate decreased to less than a third of the initial response rate, participants were exposed to a novel stimulus condition consisting of a change in one of the following: reinforcer value, reinforcer type, reinforcer amount, reinforcement schedule, or color on the computer screen, with no change used as a control. Eighty-nine percent of the participants who experienced a novel stimulus condition increased in responding

"fnint-07-00107" — 2014/1/7 — 19:37 — page 7 — #7

relative to the immediate pre-novel stimulus response rate. These results are consistent with stimulus specificity and/or dishabituation, indicating that the observed declines in responding reflect habituation.

In this section we have provided evidence that sensory reinforcement is regulated by habituation related processes. We have described data indicating that decreases in responding for visual reinforcers in rodents are well-described by six of the 10 characteristics of habituation listed by Rankin et al. (2009). We have also provided evidence that HRE occurs for sensory reinforcers in human participants as well. McSweeney and coworkers (McSweeney et al., 1996a; McSweeney and Murphy, 2000, 2009) have provided similar evidence indicating that within-session changes in responding by animals responding for food and water reinforcers can also be explained by the characteristics of habituation. Finally, as was described in the previous section, Epstein and coworkers (Wisniewski et al., 1992; Myers Ernst and Epstein, 2002; Epstein et al., 2003, 2008, 2010; Temple et al., 2007a,b, 2008b) have repeatedly provided evidence supporting HRE for food reinforcers in humans.

#### **THE QUANTITATIVE MEASURE OF HRE**

In order to more precisely characterize HRE, we have developed a method to quantify HR. Habituation assumes a declining rate of responding as a function of repeated stimulation. Our HR metric estimates the rate at which responding declines during a test session. Importantly, the HR measure is calculated so that absolute differences in response rate do not affect the HR estimate. As others (McSweeney et al., 1996a; Leussis and Bolivar, 2006) have pointed out, if differences in baseline responding are not taken into consideration, differences attributed to habituation may actually be due to baseline differences in absolute response levels.

**Figure 3A** shows data from a hypothetical test session plotted using five epochs (choice of epoch length is arbitrary). Three different hypothetical examples of habituation are shown in **Figure 3A**. HR estimates the rate at which responding declines during the test session. Calculation of this metric is a three step process.


$$\frac{\text{Percent}}{\text{Epoch}} = 100 \times \frac{\text{Rs}\_{\text{Epoch}}}{\text{TotalRs}},\tag{1}$$

where RsEpoch is the total number of responses emitted during a given epoch and TotalRs is the total number of responses that occurred during the session. This transformation normalizes the data from habituation curves that have different absolute numbers of responses and produces habituation curves with equal areas under the curve. For example, the three habituation curves shown in **Figure 3A** have different absolute levels of responding, while the habituation curves shown in **Figure 3B** all sum to 100%.

(3) Use the normalized percentage values to calculate the difference in the percent of responding between the first epoch and the minimum epoch (i.e., the epoch having the lowest percent of responses). This difference is then divided by the amount of time that elapsed between the first epoch and the minimum epoch to produce a rate measure. This is done using the following equation:

and the epoch with the lowest percentage of responding to produce the

HR measure shown in plot **(C)**. See text for details.

$$\text{HR} = \frac{(\text{Percent/Epcoh})\_{\text{First Eqn}} - (\text{Percent/Epcoh})\_{\text{Min Equach}}}{\text{Time}\_{\text{BetweenEpcoh}}},\tag{2}$$

"fnint-07-00107" — 2014/1/7 — 19:37 — page 8 — #8

where HR is the rate of habituation, (Percent/Epoch)FirstEpoch and (Percent/Epoch)MinEpoch are calculated using Eq. 1 for the first epoch and the epoch with the smallest percentage of responses respectively, and TimeBetweenEpochs is the amount of time elapsed from the midpoint of the first epoch to the midpoint of the minimum epoch. HRs for the three absolute responding curves shown in **Figure 3A** are depicted in **Figure 3C**. A HR of 1.0 means that while undergoing habituation, for every minute of elapsed session time the response rate will decrease by 1% of the total session response rate. The unit of HR is percent change in response rate per minute (Percent/Epoch/Min).

In practical application, HR can be calculated directly from the data shown in **Figure 3A** using the equation:

$$\text{HR} = 100 \times \frac{\text{Rs}\_{\text{FirstEpoch}} - \text{Rs}\_{\text{Min Expoch}}}{\text{Total}\_{\text{Rs}} \times \text{Time}\_{\text{BetweenEpochs}}},\tag{3}$$

where RsFirstEpoch represents the number of responses made during the first epoch of a session, RsMinEpoch is the number of responses made during the epoch with the fewest number of responses, TotalRs is the total number of responses made during a session, and TimeBetweenEpochs is the amount of time elapsed from the midpoint of the first epoch to the midpoint of the minimum epoch.

#### **EFFECT OF STIMULANTS ON HRE**

There is evidence that systemic administration of psychomotor stimulants, including caffeine (Sheppard et al., 2012), damphetamine (Glow and Russell, 1973a,b, 1974; Gomer and Jakubczak, 1974; Winterbauer and Balleine, 2007), methamphetamine (Gancarz et al., 2012a; Lloyd et al., 2012b), and nicotine (Chaudhri et al., 2006a; Palmatier et al., 2006; Chaudhri et al., 2007; Raiff and Dallery, 2009), enhance the primary reinforcing effectiveness of sensory stimuli. We have shown for both nicotine (Lloyd et al., in press) and methamphetamine (Gancarz et al., 2012a; Lloyd et al., in press) that the druginduced increases in responding for sensory reinforcers are accompanied by a decrease in HRE. **Figure 4A** illustrates our findings in rats administered 0.4 mg/kg nicotine, and **Figure 4B** illustrates the effects of 0.25 and 1.0 mg/kg doses of methamphetamine.

Together, the two studies illustrate the central importance of habituation in understanding stimulant effects on reinforcement. Compared to saline, neither drug simply increased the initial degree of reinforcement and then followed the same time course as saline. The higher dose of METH (1.0 mg/kg, **Figure 4B**) was the only drug condition in which the initial reinforcer effectiveness (epoch 1) was greater than that of the placebo. The impact on reinforcer effectiveness for both doses as compared to placebo increased across epochs. In contrast to the steep reduction in response rate in saline-treated rats, active responding is maintained at a relatively stable rate in those receiving 1.0 mg/kg of METH. Thus, it appears that the majority of the drug's effect at this dose was due to a reduction in HRE.

The situation is even more striking for nicotine and the lower dose of METH (**Figures 4A,B**). Neither of these drug conditions increased initial reinforcing effectiveness above that observed with

saline (epoch 1); the entire impact on reinforcer effectiveness for both METH 0.25 mg/kg and for nicotine was due to a slowing of HRE, a reduction in the loss of reinforcer effectiveness over time compared to the saline conditions.

METH increased overall responding but NIC did not. The histogram bars on the right show habituation rate (HR) (see text for explanation of HR).

These data indicate that a more detailed behavioral analysis of the within-session pattern of responding may reveal significant drug effects which are obscured by only measuring overall response rate. An important implication of these data is that stimulant drugs may increase the reinforcing effectiveness of sensory stimuli by disrupting or slowing normally occurring HRE. Stimulant-induced decreases in HRE may play a role in the abuse potential of stimulant drugs by sustaining the reinforcing effectiveness of sensory stimuli associated with drug consumption.

"fnint-07-00107" — 2014/1/7 — 19:37 — page 9 — #9

#### **DOES HRE CONTRIBUTE TO STIMULANT ADDICTION?**

In support of the hypothesis that HRE plays a role in the drug addiction processes, the ability of stimulant drugs to enhance responding for visual stimuli has been found to be an important factor in rodent drug self-administration studies where the onset (or offset) of a VS is frequently paired with drug delivery. In drug self-administration studies, visual stimuli have often been used to signal drug availability with the assumption that they do not have reinforcing properties of their own. However, at least for nicotine self-administration studies, it has been shown that the reinforcing effects of cue lights interact with the reinforcing effects of drug. In a series of experiments, Donny, Caggiula and coworkers (Caggiula et al., 2001, 2002a,b, 2009; Donny et al., 2003b; Chaudhri et al., 2006a,b, 2007; Palmatier et al., 2006) have shown that self-administration of nicotine is greatly enhanced by the response-contingent presentation of a VS. Importantly, these studies show that the effects of nicotine on the reinforcing effects of visual stimuli do not depend upon their ability to predict nicotine injections. Rather, they show that increased responding for visual stimuli is due to nicotine-induced increases in the primary reinforcing effectiveness of the visual stimuli. The results of these experiments are extremely important because many investigators do not consider that visual stimuli may have primary reinforcing effects and assume that responding for the visual stimuli that have been paired with drug to be due to conditioned reinforcing effects.

The widespread use of visual stimuli in SA procedures may be an important but largely unexamined aspect of many rodent self-administration studies. We (Gancarz et al., 2011) conducted a limited literature search in the journals (I) Psychopharmacology, (II) Physiology & Behavior, and (III) Pharmacology, Biochemistry and Behavior using the search terms (i) rat, (ii) selfadministration, and (iii) cocaine or amphetamine for the years 2007–2010. Of the 101 articles surveyed, 88 (or 87%) used a VS as a cue of drug availability/unavailability in the self-administration procedure (i.e., signaling drug delivery with onset of a house-light, cue light, or both; or flashing or colored lights; or light paired with tones or lever retraction). Surprisingly, only one study recognized use of a VS as a possible confound in interpretation of results (Keiflin et al., 2008).

Therefore, in rodent drug self-administration studies that pair drug delivery with a cue, it is not possible, without proper controls, to determine if subjects are responding to produce the light-onset, administer the drug or some combination. We hypothesize that HRE may play a role in rodent drug self-administration particularly in studies that pair drug delivery with a cue such as light-onset. As has been described above, under normal circumstances the reinforcing effectiveness of sensory reinforcers habituates rapidly. By counteracting the effects of HRE, stimulant drugs maintain the reinforcing effectiveness of visual stimuli and sustain responding. The fact that increases in drug self-administration may be due to increases in the reinforcing effectiveness of the visual cues rather than the reinforcing effectiveness of drug itself should be taken into account.

This same mechanism of disruption (slowing) of normally occurring HRE may contribute to the abuse potential of stimulant drugs such as amphetamines and nicotine in humans. Individuals habituate to sensory and social stimuli to which they are repeatedly exposed, particularly when these events are not associated with unusual or important consequences. If stimulant drugs cancel or counteract the normal process of habituation, stimuli and social interactions to which we are exposed on a daily basis continue to evoke the same excitement and rewarding effects as the first time that we encountered them. Disruption of habituation may result in a more exciting and rewarding subjective experience of the world. Indeed, the effects of abused stimulant drugs are often described as making things more rewarding than they usually are. The HRE concept and data reviewed above suggests that this is not simply an overall increase, but largely a reduction in the natural decrease in reinforcing effectiveness that occurs in the absence of stimulants. It also raises the very interesting possibility that stimulants may be particularly powerful among individuals who are prone to more rapid HRE.

#### **STIMULANT DRUGS MAY HAVE DIFFERENT EFFECTS ON SENSORY AND CONSUMABLE REINFORCERS.**

In a recent study (Gancarz et al., 2012a), we compared the reinforcing effects of light-onset and water reinforcers using a concurrent VI schedule procedure. Water-restricted rats were pre-exposed to a dark experimental chamber with two snout-poke holes. During a 16 session pre-exposure phase, responding to one hole was reinforced by 0.025 mL water presentations according to a VI 12 min schedule. In the subsequent choice phase, water reinforcement continued unchanged but responding to the alternative hole produced 5 s light-onset presented according to a VI 1 min schedule. During the choice phase, one group of animals was treated with saline and the other group was treated with 0.5 mg/kg METH. As shown in **Figure 5**, while fewer than 15% of responses in the saline group were for the light, about 30% of the METH group responses were for the light.

These results indicate two things: (1) evidenced by the rats' preference for the small less frequent water reinforcer over the more frequent sensory reinforcer, sensory reinforcers have weaker reinforcing effects than biologically important reinforcers; and (2) evidenced by METH-treated rats showing increased preference for light over saline-treated rats, methamphetamine differentially increases the reinforcing effectiveness of sensory reinforcers. A more detailed analysis of these data is presented in **Figure 6**. **Figure 6** shows responding for the sensory and water reinforcers broken down into epochs for saline- and methamphetaminetreated rats. The sensory reinforcer showed clear HRE while the water did not. While METH both increased the rate of responding and decreased the rate of habituation for the sensory reinforcer, it had no significant effect on concurrent responding for the water reinforcer.

A third group was included in the study (data not shown) that received METH injections and water reinforcement but no sensory reinforcement. In this group, responding to the nonwater producing snout-poke alternative (inactive) was recorded but had no programed consequences. For this group, METH significantly increased responding for the water alternative without changing response rate to the inactive hole. The control group indicates: (1) that drug-induced adipsia was not a factor; (2) that the response-contingent light was critical for the results; and

"fnint-07-00107" — 2014/1/7 — 19:37 — page 10 — #10

**choice between a sensory reinforcer and water in water-restricted rats.** The asterisk (\*) indicates a difference in responding between saline and METH treated rats (*p* < 0.05) Rats treated with saline responded for the sensory reinforcer <15% of the time. Rats treated with METH responded for the sensory reinforcer about 30% of the time.

with the visual stimulus shown in the left plot. The histogram plot on the right shows HRE for sensory and water reinforcers and the effects of saline and METH treatment on HRE. In the histogram, asterisks (\*) indicate an overall difference in Habituation Rate between VS and water reinforcers (*p* < 0.05), and a difference in Habituation Rate between the saline and METH treated rats receiving the VS reinforcer (*p* <0.05).

(3) that rate-dependency does not explain the absence of a significant effect of METH on responding for water (see Gancarz et al., 2012a for details). In summary, this experiment indicates that stimulant drugs such as methamphetamine may have differential effects on sensory and consumable reinforcers and that HRE may be related to this differential effect.

A possible explanation for this effect is that the methamphetamine had a greater effect on the light reinforcer because it had greater HRE. According to this idea very strong reinforcers like water (in water-restricted rats) with small HRE would be less affected by methamphetamine. Reinforcers with small HRE would have less room for stimulants to have a significant effect on HR – but it is possible they would in a longer duration session, with more opportunity for HRE.

Stimulant drugs such as amphetamine are known to induce both anorexia and adipsia in human and non-human animals (Carr and White, 1986). There is evidence that these effects may be mediated by sensory reinforcers that are concurrently available with food or water reinforcers. Carr and White (1986) suggest that stimulant treatment may "increase the tendency for an animal to approach all environmental stimuli as if they were rewarding" and that approaching alternative stimuli competes with the consumption of food and/or water, resulting in decreased consumption (anorexia or adipsia). In support of this hypothesis, they describe an experiment (Carr and White, 1986, p. 21) in which rats pretreated with amphetamine and tested in a stimulus-rich environment consumed less food than amphetamine pretreated rats tested in a barren environment. They interpreted these results as indicating that amphetamine's anorectic properties were due to potentiation of the rewarding value of stimuli other than food.

Corwin and Schuster (1993) also reported results consistent with this interpretation of amphetamine-induced anorexia. They compared the effects of d-amphetamine on choice between a sensory reinforcer andfood in rhesus monkeys. The sensory reinforcer was visual access to the laboratory in which the monkeys were housed. Using a discrete trials procedure, the monkeys chose between food and visual access. They found that lower doses of d-amphetamine increased responding for the sensory reinforcer and decreased choice of food. These results are similar to the results presented in **Figures 5** and **6**, where METH both increased respondingfor theVS and decreased HREfor the sensory reinforcer but did not affect responding for water.

To the best of our knowledge, the hypothesis that stimulants cause anorexia by increasing the reinforcing effectiveness of alternative (sensory) reinforcers has not been directly tested in humans, although there is some suggestive data. Leddy et al. (2004) tested the effects of the stimulant methylphenidate on eating pizza in obese men. They found that methylphenidate decreased the amount of pizza that was eaten. Interestingly, methylphenidate did not significantly decrease self-reports of hunger. However, there was no measure of alternative behaviors that may have been increased by methylphenidate. In another study, Perkins et al. (1995) tested abstinent and non-abstinent smokers on a concurrent variable ratio schedule procedure in which they chose between food and money reinforcers. There was an effect of smoking abstinence in a subset of the participants (dietary restrained females). These participants responded less for the food reinforcer and more for the money reinforcer after smoking. However, interpretation of these data in the context of HRE is not possible because within-session changes in responding were not reported, so the effects of smoking on HRE, if any, could not be determined.

"fnint-07-00107" — 2014/1/7 — 19:37 — page 11 — #11

# **SENSITIZATION**

Finally, a possible explanation of the stimulant-induced slowing of HRE is that the drug sensitizes the animal to the effects of the reinforcer. If we consider the rate of responding in the first epoch (before habituation occurs) as an indicator of sensitization, then there is evidence that the rats given 1.0 mg/kg METH (**Figure 4B**) were sensitized to the reinforcing effects of light-onset. The fifth behavioral characteristic of habituation (**Table 1**) states, "The less intense the stimulus, the more rapid and/or more pronounced the behavioral response decrement. Very intense stimuli may yield no observable response decrement." It follows from this characteristic that a drug-induced increase in the initial reinforcing effectiveness would also induce a slowing of HRE relative to a non-drug control. Thus it is possible that slowing of HRE is a secondary effect of a drug-induced increase in initial reinforcer effectiveness. The data showing the effects of a 1.0 mg/kg dose of METH (**Figure 4B**) on responding for a sensory reinforcer are consistent with this interpretation. This dose of METH produced a large increase in the initial rate (the first epoch in the figure) of responding as well as a slowing of HRE.

In contrast, a sensitization interpretation is not supported by the data for the 0.25 mg/kg dose of METH (**Figure 4B**) because the rate of responding in the first epoch is not different from that of the saline-treated condition. A sensitization interpretation is also not supported by the nicotine data (**Figure 4A**) which also showed a slowing of HRE with no initial increase in reinforcing effectiveness. We conclude that stimulant drugs may have separable effects on sensitization and HRE but that more moderate doses may affect HRE alone.

Future research is needed that examines within-session patterns of responding to firmly establish the relationship between the effects of stimulant drugs on the initial reinforcing effectiveness and HRE. Studies of the effects of stimulant drugs on operant responding most often report only overall average responding for test sessions. These averages are to some degree based on the assumption that the rate of operant responding is constant throughout a test session. The pre-clinical work of McSweeney and her students, and the work we have described above using light-onset as a reinforcer indicate that there are large systematic changes in within-session response rate. HRE provides a systematic approach to characterizing these withinsession changes. If our hypothesis that stimulant drugs slow HRE is correct, then it is important to analyze operant responding in a way that allows inspection of within-session changes in responding.

# **NEURAL/BEHAVIORAL MODEL OF HRE**

A conceptual model of habituation of sensory reinforcers is depicted in **Figure 7**. The relationship between DA and sensory consequences depicted in **Figure 7** is largely derived from a model described by Redgrave and Gurney (2006). The Redgrave and Gurney model describes how animals may learn new operant contingencies between responses (actions) and indifferent stimuli. Operant responses emitted by the organism produce indifferent sensory consequences. Although the stimulus itself may not be novel, at least the contingency between response and stimulus consequence is novel (i.e., the sensory stimulus is predicted by

the response). The novel sensory contingency increases DA neurotransmission in the basal ganglia which, according to the Redgrave and Gurney model, increases the probability that the animal will repeat the responses that preceded the occurrence of the sensory stimulus. Because reinforcing effectiveness is operationally defined as an increase in response rate, the Redgrave and Gurney model essentially describes a putative neural mechanism that underlies the process of reinforcement. This parsimonious account does not require reference to rewards or incentives.

Novelty and/or surprise have been shown to play an important role in the reinforcing effects of sensory stimuli. Surprise occurs because the occurrence of the stimulus is not predicted by available cues (Kamin, 1969; Blanchard and Honig, 1976; Lloyd et al., 2012b). In the model, repetitive occurrence of responsecontingent sensory consequences generates inhibitory signals in a comparator mechanism that counteracts the effects of the sensory stimulus on DA neurotransmission. The strength of the inhibitory signal is an integral of the number of previous sensory reinforcer presentations so that the ability of the inhibitory signal to cancel the effects of the response-contingent sensory stimulus on DA is directly related to how many times it has previously occurred.

The simple comparator model described in **Figure 7** is consistent with memory-based explanations of habituation (Konorski, 1963; Sokolov, 1963; Wagner, 1979). According to memorybased explanations of habituation, perceived stimuli are compared to existing memory. If the perceived sensory stimuli match memory, they are not novel and inhibit the output of the comparator. On the other hand, if the perceived stimuli do not match memory, they are novel or surprising and do not inhibit the output of the comparator. In the model described above, we have avoided the use of a memory construct and simply described the functional relationship between reinforcer repetition and the inhibitory effects of the response-contingent sensory stimulation.

As was mentioned above, the model shown in **Figure 7** is based on a more sophisticated neural model described by Redgrave and coworkers (Redgrave and Gurney, 2006; Redgrave et al., 2008, 2011) that was developed to explain the reinforcing effects of unexpected, indifferent sensory stimuli. Recently Bolado-Gomez and Gurney (2013) have incorporated a quantitative version of this model into a robotic simulation preparation which reproduces the observed behavioral patterns we identified in our published light-reinforcement experiments (Gancarz et al., 2011; Lloyd et al., 2012a). The Redgrave model indicates that sensory consequences (such as light-onset) are detected in sensory areas of the brain (superior colliculus) and cause phasic firing of DA neurons. However, DA neuron activation occurs only if the contingency between the response and indifferent sensory stimulus is unexpected. According to the Redgrave model, increased DA transmission in the basal ganglia causes re-selection of recently emitted responses and thus repetition of the behavior that produced the response-contingent sensory stimulus.

While there is evidence that the neural circuitry described by Redgrave and coworkers underlie increased DA neurotransmission in response to unpredicted response-contingent stimuli, less

"fnint-07-00107" — 2014/1/7 — 19:37 — page 12 — #12

is known about the neural circuitry that may mediate inhibition of the effects of the sensory reinforcer on DA neurotransmission. The neural structures that cancel out the effects of response-contingent sensory stimuli on DA neurotransmission are largely unknown. Redgrave et al. (2008) have suggested the habenula to be the source of the canceling signal. The lateral habenula may be particularly

important because it provides a convergence for neural information from the basal ganglia and limbic forebrain and modulates activation of DA neurons in response to sensory stimuli (Bianco and Wilson, 2009).

The model presented in **Figure 7** greatly over-simplifies the processes underlying HRE; it does, however, serve to highlight

"fnint-07-00107" — 2014/1/7 — 19:37 — page 13 — #13

two separate neural processes that may underlie changes in the rate of HRE. According to the conceptual model in **Figure 1**, HRE would be *increased* by processes that potentiate inhibition of the effects of previously experienced sensory consequences on DA neurotransmission and by processes that directly *decrease* DA neurotransmission. Conversely, HRE would be decreased by processes that interfere with inhibition of the effects of sensory consequences on DA neurotransmission and by processes that increase DA neurotransmission.

The model described in **Figure 7** is consistent with the declining pattern of responding observed in rats responding to produce light-onset (characteristic 1). Spontaneous recovery (characteristic 2) can also be understood within the general framework of the model if it is assumed that the integrator in **Figure 7** is "leaky." In this case, inhibitory output from the integrator into the comparator should diminish with time. A leaky integrator would predict spontaneous recovery of responding after periods of no responding. The degree of spontaneous recovery would be a function of how rapidly the integrator "leaked" and the passage of time. Conversely, individuals with very leaky integrators may show very little habituation.

The framework provided by the model outlined in **Figure 7** indicates that drugs that increase DA neurotransmission, such as psychomotor stimulants, should decrease HRE, and that drugs that decrease DA neurotransmission should increase HRE. Presumably, changes in the integrator/comparator function would also lead to changes in downstream DA neurotransmission. A weakness of the model is the lack of specificity with regard to the integrator/comparator function. Research is needed to better understand the neural mechanisms that underlie the integrator/comparator function hypothesized by the model. It seems likely that individual differences in HRE associated with behavioral problems such as those described in the clinical significance section below are due to a dysfunction of the hypothesized integrator/comparator process, and that downstream changes in DA function may be secondary to these effects.

#### **CLINICAL SIGNIFICANCE OF HRE**

Every day, we experience a wide range of sights, sounds, tastes, smells, and tactile stimuli that influence our behavior via operant learning mechanisms. The literature presented above suggests that the reinforcing effectiveness of these sensory reinforcers is subject to habituation. Thus, the rate of HRE may play a critical role in behavioral regulation, and abnormal HRE may result in behavioral dysregulation. Dysfunction of HRE can come in two forms. Abnormally slow HRE would result in excessive or persistent responding to produce sensory consequences and, conversely, abnormally rapid HRE would cause premature cessation of responding for sensory reinforcers. In extreme cases dysfunctional HRE may affect the occurrence and severity of severalforms of psychopathology. In this section, we will discuss clinical conditions that may be characterized by slowed and accelerated HRE.

#### **SLOW HABITUATION**

Examples of clinical disorders that may be related to slow-HRE are ASD and obesity. There is good evidence that the stereotypies associated with ASD are operant responses maintained by sensory reinforcement (Lovaas et al., 1987; Rapp and Vollmer, 2005a,b; Cunningham and Schreibman, 2008). From an HRE perspective, the repetitive nature of these operant responses indicates a failure of normally occurring HRE, such that the individual is continually reinforced by sensory properties of the stereotypic behavior (e.g., rocking, head banging, etc.). That said, no study to date has approached ASD-related stereotypies from an HRE framework.

In contrast, there is a large body of work suggesting that obesity is associated with diminished HRE. More than 35% of adults and 15% of youth in the U.S. are obese (Ogden et al., 2012), and obesity is now the second leading cause of preventable death in the U.S. Slow HRE would result in maintenance of food's reinforcing properties, leading to greater caloric intake. As was described in the introduction, Epstein and coworkers have reported that HRE of food is reduced among obese compared to non-obese people (Temple et al., 2007b; Epstein et al., 2008). In a recent longitudinal study of lean children, those who exhibited slower habituation of operant responding to produce small amounts of food demonstrated greater gains in BMI over the subsequent year (Epstein et al., 2011). These prospective data provide preliminary evidence that individual differences in HRE contribute to the development of obesity, rather than simply covary with obesity.

The HRE framework suggests several ways to impact the reinforcing effectiveness of food. Specifically, the characteristics of habituation (**Table 1**) suggest that interventions that speed HRE of food reduce food intake. For example, HRE theory predicts that varying reinforcers (types of food) should slow HRE, and that repeatedly presenting the same reinforcer should accelerate HRE. Indeed, laboratory experiments demonstrate that increasing dietary variety decreases the rate of habituation on food consumption, while increasing hedonics and salivation, and is associated with increased energy intake (Wisniewski et al., 1992; Temple et al., 2008a). Conversely, Epstein et al. (2011) compared the effects of eating the same, similar, or a variety of food and demonstrated a reduction of energy intake in test groups eating foods with the same or similar characteristics. In a separate study, Epstein et al. (2013) demonstrated long-lasting habituation by assigning the same food more often (daily versus once per week), which resulted in less energy intake over the course of the 5-week study.

These results illustrate how HRE can be applied to develop dietary programs. Several popular diets appear consistent with the principle of reducing dietary variety (e.g., Atkin's diet, Slim-Fast®). In addition, a clinical trial is presently underway to further assess the effects of reinforcer variety in children's diet. This behavioral intervention is attempting to define the optimal interval for stimulus and variety reduction in diet that will facilitate long-term habituation of calorically dense foods in an effort to address the growing problem of pediatric obesity (ClinicalTrials.gov number NCT01208870).

Another interventional approach to decreasefood consumption is to strengthen the reinforcing value of non-food sensory stimuli via stimulant medication. An estimated 15% of Americans take dietary supplements for weight loss (Blanck et al., 2007). Many of these dietary supplements contain stimulant components and are marketed as "boosting energy" and "staving off hunger." As described in section "Effect of Stimulants on HRE," stimulants

"fnint-07-00107" — 2014/1/7 — 19:37 — page 14 — #14

have the potential to induce anorexia and adipsia in humans and animals. Recent work indicates that stimulants may achieve this effect, not by suppressing appetite *per se*, but by increasing the effectiveness of alternative non-consumable reinforcers. In animals, Gancarz et al. (2012a) demonstrated that methamphetamine increases the reinforcing effectiveness of sensory stimuli (light reinforcement) more than concurrently available water reinforcers. We hypothesize that stimulants increase the relative reinforcing value of alternative non-consumable reinforcers which decreases the relative reinforcing effectiveness of food.

Pre-clinical data demonstrating the effects of stimulants on HRE provides insight into the mechanisms of regulation of eating behavior. While stimulant drugs have undesirable side-effects including a potential for abuse, their pre-clinical use allows elucidation of the mechanism by which they decrease overall caloric intake and can suggest behavioral interventions which can be safely implemented. For example, a potential diet may recommend both increasing the variety of alternative (non-food) reinforcers a person experiences (thereby decreasing habituation of non-eating behaviors), while concurrently decreasing the variety of food reinforcers (thereby increasing habituation of eating). The HRE framework predicts that this sort of manipulation would both slow HRE for alternative (non-food) reinforcers and accelerate HRE for food reinforcers.

#### **RAPID HABITUATION**

Clinically, rapid HRE could lead to a lack of responsiveness to typical reinforcers in one's environment. We hypothesize that rapid HRE is present in ADHD, which is characterized by developmentally inappropriate levels of inattention and/or hyperactivity and impulsivity that creates problems in multiple settings (American Psychiatric Association, 2013). As described below, HRE may inform our understanding of both ADHD psychopathology and the mechanisms of action of the leading evidence-based treatments for the disorder (i.e., behavior therapy and stimulant medication).

Despite the central role of reinforcement in theories of ADHD (Haenlein and Caul, 1987; Sagvolden et al., 2005), there is considerable controversy regarding the exact nature of the dysfunction (Luman et al., 2005; Sonuga-Barke, 2011). From an HRE perspective, poor stimulus control by environmental and social stimuli in children with ADHD may be due to accelerated habituation to reinforcers, rather than a static reinforcement dysfunction, such as an "elevated reward threshold" (Haenlein and Caul, 1987).

The HRE framework developed here provides a relatively novel explanation of ADHD-related deficits. However, this hypothesis has gone largely untested. One exception is the work of Douglas and coworkers (Iaboni et al., 1997) examining the impact of monetary reinforcement on behavior (pressing one of several buttons to turn off a response box light) and heart rate, which tends to accelerate under reinforcement conditions. Douglas et al. found that children with ADHD exhibited faster habituation of heart rate responses to reinforcement than did typically developing children. Although behavioral responding did not show the same pattern, there were only a small number of testing blocks, and each 2-min block was followed by a break, which may have markedly reduced the ability to observe behavioral habituation.

Extensions of this type of work with a simple but boring task, modest reinforcers, and longer testing periods may provide an excellent laboratory analog of real-world conditions under which children with ADHD have great difficulty regulating their behavior (e.g., completing seatwork or homework for 20–60 min). Such a paradigm would also be excellent for testing the degree to which HRE is slowed/accelerated for sensory (and perhaps other) reinforcers in ADHD. Although not presently used to investigate HRE, such a paradigm is already widely used in the ADHD literature. Continuous performance tasks (CPTs) require a response (button press) to infrequent, briefly presented target stimuli over a long period of time (typically 10–30 min), and a decrease in target detection over time is interpreted as a measure of sustained attention, or vigilance. Children with ADHD exhibit a steeper decrease in target detection over time than do typically developing children (e.g., Huang-Pollock et al., 2006; for a review, see Huang-Pollock et al., 2012), which is interpreted as a deficit in sustained attention. However, these results could also be explained by habituation mechanisms. If children with ADHD habituate more rapidly to reinforcers, then the sensory stimuli presented on the computer screen during a CPT would lose their ability to regulate behavior more quickly, leading to a decrement in performance over time.

Mackworth (1969), an early researcher in the study of vigilance tested several predictions based on the characteristics of habituation and concluded in her 1969 text*Vigilance and Habituation* that "the vigilance decrement is a particular example of the process of habituation" (p. 185); she went on to say "These changes probably reflect the reduction in the amount of attention paid to the repetitive stimuli. The changes can be regarded as a reduction in either the quality or quantity of *observing responses* made toward the events of the task" (p. 186). Interestingly, this work has to our knowledge never been applied to the study of ADHD, and the role of HRE in sustained attention receives little consideration in the broader cognitive literature (cf., Ariga and Lleras, 2011; Helton and Russell, 2011).

The HRE hypothesis also addresses the disconnect between neurobiological models and psychological models of reinforcement: "The scarcity of studies testing neurobiological predictions is explained partly by a lack in knowledge of how to test some of these predictions in humans" (Luman et al., 2010). Conversely, psychological models "offer few testable experimental predictions" and are not integrated with neurobiological mechanisms (p. 745). The HRE hypothesis provides a conceptual and empirical bridge between contemporary neural and behavioral models of reinforcement. This is particularly important for understanding the role of reinforcement mechanisms in the leading psychosocial (behavior therapy) and pharmacological (psychostimulant) ADHD treatments (Volkow et al., 2005, 2009; Tripp and Wickens, 2012).

Contingency management approaches, which include the systematic application of reinforcement to enhance the rate of a targeted behavior, are best practice interventions for youth with ADHD. Reinforcement typically involves the contingent presentation of sensory stimuli (e.g., stickers, points, and praise). In these interventions, teachers and parents are typically advised to consistently reinforce the child after every instance of a targeted

"fnint-07-00107" — 2014/1/7 — 19:37 — page 15 — #15

behavior (i.e., continuous reinforcement; FR1). However, parents often report that reinforcement works initially, but then loses its efficacy over time. As some leading researchers in the field report, "A reinforcer loses its effect when it is given in excessive amounts ... " (Kazdin, 2005) and "Children often become satiated quickly with rewards ... resulting in a loss of motivating power as a behavioral-change tool" (Barkley, 1997). Despite this commonly reported clinical concern regarding diminished reinforcing effectiveness over time, these decreases have not been systematically studied. Clinicians often address diminished reinforcing effectiveness by recommending the use of a menu of different reinforcers or introducing new rewards. Paralleling the advances in obesity research noted above, we believe that such treatment programs may be improved through explicit study of the problem of diminishing reinforcer effectiveness within an HRE framework.

The HRE model provides novel avenues for slowing the rate of HRE thereby increasing the rate of a desired behavior. In one of the few studies to have investigated HRE in human participants, Kenzer et al. (2013) were able to increase response rates with a dishabituation manipulation. More studies are needed to investigate how the principles of HRE can be used to improve behavioral interventions for children with ADHD. For example, shorter training sessions interspersed with other activities of recreation (allowing spontaneous recovery) and infusing existing reward menus with theory-based manipulations to capitalize on stimulus specificity (i.e., vary the reinforcers more systematically to proactively prevent habituation, rather than wait for a reinforcer to lose its effectiveness before switching reinforcers) are possible ways to slow habituation in children with ADHD to improve behavior and academic outcomes.

The application of an HRE framework to the understanding of ADHD pathology and treatment is novel, and there are no published studies in this area. The HRE model offers an interpretation of clinical observations as well as clear directions for enhancing treatment. For example, the HRE hypothesis predicts that continuous reinforcement of a behavioral contingency with a single sensory stimulus would be counterproductive because the child would habituate to the reinforcer, and it would lose its effectiveness to control behavior. There is some evidence from pre-clinical studies that the rate of responding for a sensory reinforcer such as light-onset is greater and the rate of habituation is slower when the reinforcer is presented less frequently (see section "Experimental Analysis of HRE with a Light Reinforcer"). These findings are consistent with the principles of HRE discussed in this paper, and provide an explanation for the diminished reinforcing effectiveness over time of contingency management programs used during behavioral intervention for children with ADHD.

In addition to behavioral intervention programs, children with ADHD are often treated with stimulant medication. Experts estimate about 60% of children with ADHD are treated with some type of stimulant medication, and overall about 5% of all children ages 6–18 use prescription stimulant medication (Zuvekas and Vitiello, 2012). Why is stimulant medication used so pervasively and how is it effective in treating children with ADHD? In rodents, methylphenidate blocks the DA transporter and increases DA overflow in the nucleus accumbens and dorsal striatum (Kuczenski and Segal, 1997; Segal and Kuczenski, 1999). Consistent with animal work, Volkow et al. (2002) demonstrated that intravenous stimulant administration in humans increases DA receptor stimulation. Volkow et al. (2004) also demonstrated that oral methylphenidate increased extracellular DA on PET scan in healthy subjects.

An HRE framework provides possible explanations for both ADHD-related impairments and the effectiveness of stimulant medication in the treatment of ADHD. Decreased DA neurotransmission in response to reinforcing stimuli could lead to more rapid HRE causing the child to struggle with maintaining attention and appropriate behavior in school. By blocking DA transporters and preventing DA reuptake, stimulant medication may sustain the effects of reinforcer-induced DA release for longer time intervals, thus slowing HRE. Slowing of HRE would increase stimulus control by social and non-social stimuli, resulting in improved behavior and academic performance. Artificially increasing DA neurotransmission with stimulants slows rapid-HRE in ADHD individuals and maintains responsiveness to reinforcers, which leads to improvements in behavior and academic achievement.

# **CONCLUSIONS AND FUTURE DIRECTIONS**

We have hypothesized that HRE is a fundamental property of reinforcers, and that reinforcer effectiveness is dynamic. Preclinical animal research has almost exclusively focused on the study of powerful consumable reinforcers which exhibit slow-HRE. Pre-clinical researchers have largely used deprivation and other manipulations that mask the effects of habituation. It is important for pre-clinical animal research to go beyond consumable reinforcers. Although there is a large older literature about light reinforcement (for reviews see: Lockard, 1963; Kish, 1966; Berlyne, 1969; Tapp, 1969; Eisenberger, 1972), it is not currently an active area of research. Characterization of the reinforcing effectiveness of a range of repeatedly presented reinforcers, including modest but ubiquitous sensory stimuli, should be an objective of future research. Reinforcers that exhibit rapid-HRE such as lightonset are weak and transient and more difficult to study than strong consumable reinforcers such as food and water. However, it is arguable that weak and transient reinforcers such as light-onset with rapid-HRE make up the majority of reinforcers that regulate day-to-day behavior.

The 10 behavioral characteristics of habituation listed in **Table 1** make testable predictions about operant responding, and research is needed to further test these predictions, particularly in humans. Some of these predictions are surprising. For example, increasing repetition by increasing reinforcer frequency is predicted to decrease response rate. This prediction holds for fast-HRE-reinforcers (see above), but a large body of research with slow-HRE-reinforcers indicates that increasing reinforcing frequency increases response rate. Parametric studies are needed to better understand the dynamic relationship between HRE, reinforcement frequency and response rate.

We have presented evidence that psychomotor stimulant drugs disrupt (slow) normally occurring HRE. This finding has at least two important implications. First, it provides an explanation

"fnint-07-00107" — 2014/1/7 — 19:37 — page 16 — #16

of the reinforcing effects of stimulant drugs on behavior. The effects of abused stimulant drugs have been described as "making things more rewarding than they usually are.2" Stimulant-induced increases in the duration of the reinforcing effects of environmental and social stimuli may underlie this subjective perception. As discussed above, the primary reinforcing effects of cues play an important role in rodent drug self-administration studies, and normally occurring HRE is disrupted by stimulant drugs.

Second, it appears that stimulant drugs may have larger effects on rapid-HRE-reinforcers than slow-HRE-reinforcers. This differential effect predicts a shift in relative preference toward the rapid-HRE-reinforcers. As has been discussed this type of preference shift may underlie the anorectic effects of stimulant drugs. Studies are needed which examine the effect of stimulant drugs on concurrent responding for slow-HRE- (e.g., food or water) and rapid-HRE-reinforcers (e.g., purely sensory stimuli) in both human and non-human subjects to determine the validity of this assertion. For example, in human subjects simple experimental paradigms could measure stimulant effects on within-session decreases in both the reinforcing effectiveness and subjective liking of a reinforcing stimulus (e.g., small amount of food, picture on a computer screen) and include stimulus specificity, dishabituation, and spontaneous recovery tests.

A method for quantitatively measuring the speed of HRE was described. The HR metric is calculated so that absolute differences in response rate do not affect the estimated HR. Future research using this technique may produce estimates of the rate of habituation that are quantitatively comparable across a variety of test situations. Using this measure to index the rate of HRE across a variety of reinforcing stimuli may be particularly important in human studies and clinical settings where a large variety of reinforcers are used.

A conceptual model of HRE was described which has two components, an integrator/comparator component reflecting habituation and a DA neurotransmission component reflecting probability of response repetition. The DA component is welldescribed by Redgrave and Gurney (2006). In contrast, the neural basis of the integrator/comparator component (where habituation takes place) is unknown. Understanding the neural basis of the integrator/comparator component of the model is important because individual differences in this component may underlie behavioral disorders such as obesity and ADHD which may be caused by abnormal slow-HRE or abnormal rapid-HRE, respectively.

Finally, the HRE concept may provide important insights into the etiology and treatment of behavioral disorders such as obesity and ADHD. Abnormal HRE due to genetic and/or environmental factors may underlie some clinical disorders. For example, recent research indicates that slow-HRE is predictive of obesity, and manipulations that accelerate HRE for food reinforcers may be important treatment components. Conversely, ADHD may reflect accelerated HRE, leading to poor stimulus control, attention, and persistence. The HRE framework suggests specific interventions such as shorter task periods (spontaneous recovery), varying the reinforcer (stimulus specificity), and changing the background context (dishabituation) that can be put to work to slow the loss of reinforcer effectiveness and improve behavioral regulation.

# **ACKNOWLEDGMENTS**

This research was supported by NIDA grants DA10588 and DA026600 awarded to Jerry B. Richards and by NIMH grant MH069434 awarded to Larry W. Hawk. NIDA and NIMH had no role other than financial support. All authors contributed significantly and have approved the manuscript. No authors have any conflict of interest, real, or apparent. We would like to thank Leah Militello, Gregory A. Fabiano, Suzanne H. Mitchell, Eric S. Murphy, and Frances K. McSweeney for their comments during manuscript preparation.

# **REFERENCES**


"fnint-07-00107" — 2014/1/7 — 19:37 — page 17 — #17

<sup>2</sup>When asked to explain why cigarette smoking was addictive on the National Public Radio program Science Friday (September 7, 2012), Dr. Nora Volkow, Director of the National Institute on Drug Abuse, stated, "When you are smoking, everything that is around you is much more salient, much more exciting," and smoking "makes things more rewarding than they usually are."


rats: effects of methamphetamine. *Exp. Clin. Psychopharmacol.* 20, 16–27. doi: 10.1037/a0025701


"fnint-07-00107" — 2014/1/7 — 19:37 — page 18 — #18


"fnint-07-00107" — 2014/1/7 — 19:37 — page 19 — #19


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 14 September 2013; accepted: 19 December 2013; published online: 09 January 2014.*

*Citation: Lloyd DR, Medina DJ, Hawk LW, Fosco WD and Richards JB (2014) Habituation of reinforcer effectiveness. Front. Integr. Neurosci. 7:107. doi: 10.3389/fnint.2013.00107*

*This article was submitted to the journal Frontiers in Integrative Neuroscience.*

*Copyright © 2014 Lloyd, Medina, Hawk, Fosco and Richards. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

"fnint-07-00107" — 2014/1/7 — 19:37 — page 20 — #20

# The mechanism of dishabituation

# *Genevieve Z. Steiner\* and Robert J. Barry*

*Centre for Psychophysics, Psychophysiology, and Psychopharmacology, Brain & Behaviour Research Institute, School of Psychology, University of Wollongong, Wollongong, NSW, Australia*

#### *Edited by:*

*Catharine H. Rankin, University of British Columbia, Canada*

#### *Reviewed by:*

*Hao Zhang, University of Alberta, Canada Nash Boutros, Wayne State University, USA*

#### *\*Correspondence:*

*Genevieve Z. Steiner, Centre for Psychophysics, Psychophysiology, and Psychopharmacology, Brain & Behaviour Research Institute, School of Psychology, University of Wollongong, Wollongong, NSW 2522, Australia e-mail: genevieve\_steiner@ uow.edu.au*

The dual-process theory of habituation attributes dishabituation, an increase in responding to a habituated stimulus after an interpolated deviant, to sensitization, a change in arousal. Our previous investigations into elicitation and habituation of the electrodermal orienting reflex (OR) showed that dishabituation is independent of sensitization for indifferent stimuli, arguing against dual-process theory's explanation. However, this could not be tested for significant stimuli in that study, because sensitization was confounded with incomplete resolution of the preceding OR. This study aimed to clarify the mechanism of dishabituation for significant stimuli by extending the stimulus onset asynchrony (SOA) beyond the time required for the phasic response to resolve. Participants completed an auditory dishabituation task with a random SOA of 13–15 s while their electrodermal activity was recorded. The stimulus sequence was 10 standards, 1 deviant, 2–4 standards; counterbalanced innocuous tones. Two counterbalanced conditions were used: silently count all stimuli (significant) and no task (indifferent). Skin conductance responses (SCRs) and pre-stimulus skin conductance levels (SCLs) both decremented over trials 1–10. In both conditions, SCRs showed response recovery and dishabituation, indicating habituation, and post-deviant SCL sensitization was apparent. Across all trials, phasic ORs were dependent on the pre-stimulus SCL (arousal level); this did not differ with condition. Importantly, dishabituation was independent of sensitization for both conditions. Findings indicate that sensitization, the *change* in state, is a process separate from phasic response resolution, and that arousal consistently predicts OR magnitude, *including* the dishabituation response.This argues against dual-process theory's explanation, and instead suggests that dishabituation is a disruption of the habituation process, with magnitude determined by the current arousal level.

**Keywords: dishabituation, orienting reflex, counting, electrodermal activity, habituation, sensitization**

# **INTRODUCTION**

The process of habituation, a simple form of learning where responding to a repetitious stimulus decreases, has been documented in a wide range of organisms (from single-celled animals to primates) and is thought to allow an organism to reflexively filter out irrelevant information. Habituation is unique from other decrementing processes such as fatigue or refractory periods, as it can be interrupted by a change in stimulation (Thompson et al., 1979). For a decrementing response pattern to be correctly classified as habituation, Thompson and Spencer (1966), and more recently, Rankin et al. (2009), suggest that *response recovery*, an enhanced response to a change-stimulus (deviant), and *dishabituation*, a post-deviant increase in response to the habituated stimulus, should be evident. This paper will focus on clarifying the mechanism underpinning the latter of these criteria, dishabituation, in the context of the orienting reflex (OR) captured by the human electrodermal response.

Over the past century, there have been several theories on elicitation and habituation of the OR, the adaptive mechanism responsible for directing an organism's attention toward environmental changes (Barry, 2009). Arguably, the two most prominent of these theories are Sokolov's (1960, 1963a,b) neuronal-model comparator theory, and Groves and Thompson's (1970) dual-process

theory. Sokolov theorized that an OR was elicited when an incoming stimulus did not match a neuronal representation of the environment, and this was extensively explored with habituation of the phasic electrodermal response (galvanic skin response, now the skin conductance response; SCR). Sokolov's model also incorporated a response amplifier, where the arousal state of the organism, measured by the slow-changing skin conductance level (SCL; Barry and Sokolov, 1993), determined the phasic response. Dual-process theory outlines two hypothetical processes that reflect the novelty of a stimulus: habituation (H), a pathway-specific decremental process; and sensitization (S), a state process that increases to novel stimuli. These two processes interact to determine OR magnitude, with the S-process regulating the outcomes of the H-process and serving as a response amplifier.

In addition to documenting decreases in OR magnitude, Sokolov (1963b) also noted that ORs increased to task-relevant "*significant*" stimuli, relative to task-irrelevant "*indifferent*" stimuli, and habituated more slowly in the context of heightened arousal. Although several studies have confirmed this finding (Barry, 2004, 2009; Steiner and Barry, 2011), neither Sokolov's neuronal model nor dual-process theory offer a theoretical mechanism to generate the arousal increases associated with

stimulus significance. Alternatively, Maltzman's (1979a,b, 1990) voluntary OR concept can predict the increased arousal and phasic responding related to significant stimuli by providing a mechanism of cortical activation. Maltzman argued that individuals generate a "cortical set" related to a range of factors including task instructions, prior learning, etc., and the voluntary OR is the outcome of the cortical activation associated with the individual's cortical set. The current study aimed to test aspects of both Sokolovian and dual process theories in the context of indifferent and significant stimuli.

Both the neuronal-model comparator theory (Sokolov, 1963b) and dual-process theory (Groves and Thompson, 1970; outlined more recently in Thompson, 2009) make different predictions about the mechanism of dishabituation, with Sokolov suggesting that dishabituation is a disturbance in the habituation process, and dual-process theory predicting that dishabituation is a superimposed process of sensitization (an increase in state level/arousal). In our previous investigation into the mechanism of dishabituation (Steiner and Barry, 2011), we tested dual-process theory's unique prediction: that dishabituation reflects nothing more than the sensitization process, a *change* in arousal. We found that dishabituation was independent of sensitization for indifferent stimuli, a finding that argues against dual-process theory's mechanism. In that study, however, the same conclusion could not be drawn for significant stimuli: the phasic response to the deviant had not resolved before the dishabituation trial, and subsequently interfered with measurement of the sensitization process. The aim of the current study was to continue along the same line of investigation used in Steiner and Barry (2011) by extending the stimulus onset asynchrony (SOA) long enough to allow resolution of the deviant response preceding the dishabituation trial.

Following Steiner and Barry (2011), we made a similar set of predictions. We hypothesized that SCR would demonstrate habituation: decrement with stimulus repetition, response recovery to an interpolated deviant, and dishabituation to the re-presentation of the habituated stimulus, regardless of stimulus significance. It was also predicted that for significant stimuli, SCRs would be enhanced and decrement more slowly than for indifferent stimuli.

Pre-stimulus SCLs (arousal level) were also expected to decrement with stimulus repetition and be enhanced for significant stimuli. We expected sensitization, apparent as an increase in arousal, to follow the deviant stimulus and that this process would be independent of the deviant responsefor *both* indifferent and significant stimuli. Again following Steiner and Barry (2011), it was hypothesized that dishabituation would be independent of sensitization, this time for both conditions, a prediction which, if shown to be true, would argue against the dual-process theory's mechanism of dishabituation. Also, and in line with Sokolov's assertion that the current arousal level amplifies the phasic response, it was predicted that SCR would be dependent on the pre-stimulus SCL.

# **MATERIALS AND METHOD**

#### **PARTICIPANTS**

Twenty-four undergraduate students participated in this study in return for course credit (age: 18–25 years, 23 right-handed, 14 males). All provided informed consent prior to participating, and were free to withdraw at any time without penalty. Participants self-reported no use of psychotropic medication, and no neurological or psychiatric illnesses. Self-reports also indicated that participants had refrained from psychoactive substances for at least 12 h and from tea, coffee, alcohol, and cigarettes for at least 2 h prior to testing. All participants had normal or corrected-to-normal vision and self-reported normal hearing.

#### **PROCEDURE**

Participants completed a demographic and screening questionnaire, were fitted with electrodermal recording apparatus, seated in an air-conditioned room 600–800 mm in front of a 19-- Dell LCD monitor (REV A00) and instructed to fixate on a 10 mm × 10 mm gray cross displayed in the center of a black background. Acoustic stimuli were delivered binaurally through Sony MDR V700 circumaural stereo headphones, and consisted of 1000 and 1500 Hz tones, each of 50 ms duration (15 ms rise/fall time), 60 dB SPL, with a random SOA of 13–15 s. The stimulus sequence included 10 tones of one frequency (standards), a *deviant* tone of a different frequency, followed by 2–4 standards; the standard/deviant frequencies were counterbalanced between subjects. All participants completed two counterbalanced conditions presented approximately 3 min apart: Indifferent, where participants were instructed that there was "no task in relation to the sounds"; and Significant, where participants were directed to "silently count the sounds and report to the researcher at the end of the experiment". This procedure was approved by the joint South Eastern Sydney/Illawarra Area Health Service and University of Wollongong Health and Medical Human Research Ethics Committee.

#### **MATERIALS AND APPARATUS**

Electrodermal data were recorded from the distal volar surface of digits II and III of the non-dominant hand using sintered silver/silver-chloride (Ag/AgCl) electrodes, filled with isotonic electrode paste of 0.05 M NaCl in an inert ointment base. Skin conductance was sampled using a constant voltage device (UFI Bioderm model 2701) at 0.5 V. The DC-coupled skin conductance output was sampled at 1000 Hz using a Neuroscan Synamps 2 digital signal-processing system and Neuroscan 4.3.1 Acquire software, and stored on a Dell Optiplex 755 computer. Stimulus presentation was controlled by a similar linked stimulus computer using Neurobehavioral Systems Inc. Presentation V 13.0 Build 01.23.09 software.

#### **DATA EXTRACTION**

Raw data were band-pass filtered (0.1–3 Hz, zero-phase shift, 24 dB/Octave) and epoched offline 1 s pre- to 13 s post-stimulus using Neuroscan 4.3.1 Edit Software. An average of 1 s of immediately pre-stimulus activity was taken as a measure of SCL. Using the linear detrend function in Neuroscan on each trial, pre-response levels (1 s pre- to 1 s post-stimulus onset) were linearly extrapolated to compensate for falling baselines, following Barry et al. (1993). Each phasic response (with onset latency 1–3 s poststimulus onset, following Barry, 1990) was quantified for each

subject and each trial, as the difference between the extrapolated baseline and the maximum value of the subsequent peak (see **Figure 1**). SCRs were square-root transformed to reduce skew (Barry and Sokolov, 1993; Barry, 2004). Trials that contained outliers (such as non-stimulus related responses) were removed and replaced by an average of the trials preceding and succeeding the outlier trial.

#### **STATISTICAL ANALYSES**

The design was a within-subject study of response decrement, response recovery, dishabituation, and sensitization for each condition. MANOVAs assessed the effect of condition across all 13 trials, separately for SCRs and SCLs. Separate MANOVAs were used to examine response decrement over trials 1–10 for both SCR and SCL with factors of condition (Indifferent vs. Significant) and trials (1–10); within trials, linear and quadratic trends were assessed. For SCR only, separate MANOVAs were carried out to assess response recovery and dishabituation, again with factors of condition (Indifferent vs. Significant) and trials (response recovery: 10 and 11; dishabituation: 10 and 12). MANOVA examined pre-stimulus SCL deviant sensitization with a comparison of trial 12–11 for both conditions. All *F*-tests are reported with (1, 23) degrees of freedom unless otherwise specified.

No Bonferroni-type α adjustment was required as contrasts were planned, and the number of contrasts did not exceed the degrees of freedom for effect (Tabachnick and Fidell, 1989). The violations of sphericity assumptions often associated with repeated-measures analyses do not affect single degree of freedom contrasts, so Greenhouse–Geisser-type correction was not necessary (O'Brien and Kaiser, 1985).

To test whether the post-deviant increase in SCL (deviant sensitization) was true sensitization (i.e., was not related to the incomplete resolution of the deviant response), a bivariate correlation compared the change in SCL (trial 12 minus 11) to the SCR at trial 11. To test dual-process theory's unique explanation of dishabituation, a further correlation was calculated that compared the extent of dishabituation in the phasic OR (SCR trial 12 minus 10) to the sensitized arousal from the deviant stimulus (to ensure a consistent comparison with SCR, SCL was also calculated as trial 12 minus 10). To test Sokolov's assertion that arousal is a response amplifier, SCRs were correlated with pre-stimulus SCLs across trials and subjects. To see how this changed over trials, separate correlations were also carried out for each trial. A multiple regression was also conducted to examine the OR determinants outlined by Sokolov, dual-process theory, and Maltzman (i.e., novelty, significance, and arousal) as predictors of SCR. As SCR was expected to reduce with novelty over trials, novelty was entered into the multiple regression as the reciprocal of each trial number (e.g., trial 10 was entered as 1/10). One-way tests were utilized for all analyzed predictions.

### **RESULTS**

Data from all participants were included as all reported the correct number of stimuli for the significant condition at the end of the condition.

#### **SKIN CONDUCTANCE RESPONSES**

**Figure 2** shows the grand mean raw SCR waveforms across all subjects and trials, separately for each condition. A condition main effect is apparent, with larger mean baselined SCRs for the significant compared to the indifferent condition across all trials (significant <sup>&</sup>gt; indifferent: *<sup>F</sup>* <sup>=</sup> 29.87, *<sup>p</sup>* <sup>&</sup>lt; 0.001, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.56).

R1 = R2, in terms of both amplitude and peak latency.

**Figure 3** illustrates the mean SCRs across all 13 trials separately for each condition. Across condition, response decrement was apparent with SCRs showing a linear decrease and plateau over trials 1–10 (linear trials: *<sup>F</sup>* <sup>=</sup> 91.60, *<sup>p</sup>* <sup>&</sup>lt; 0.001, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.80; quadratic trials: *<sup>F</sup>* <sup>=</sup> 65.54, *<sup>p</sup>* <sup>&</sup>lt; 0.001, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.74). The quadratic trend differed with condition (quadratic trials × significant > indifferent: *<sup>F</sup>* <sup>=</sup> 7.35, *<sup>p</sup>* <sup>=</sup> 0.012, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.24), suggesting that the indifferent responses continue to decline more systematically than the responses to the significant stimuli. Significant SCRs were also larger than SCRs to indifferent stimuli across the first 10 trials (significant <sup>&</sup>gt; indifferent: *<sup>F</sup>* <sup>=</sup> 19.27, *<sup>p</sup>* <sup>&</sup>lt; 0.001, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.46).

**Figure 3** also shows that response recovery was apparent, with larger SCRs for the deviant (trial 11) than the preceding trial 10 (10 <sup>&</sup>lt; 11: *<sup>F</sup>* <sup>=</sup> 7.39, *<sup>p</sup>* <sup>=</sup> 0.012, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.24). SCRs were also larger over trials 10 and 11 for the significant than indifferent stimuli (significant > indifferent: *F* = 7.77 *p* = 0.010, η2 <sup>p</sup>= 0.25). There was no trial by condition interaction, indicating that the change in response from 10 to 11 did not differ with condition.

Dishabituation was evident, with larger SCRs at trial 12 compared to 10 (i.e., 10 <sup>&</sup>lt; 12: *<sup>F</sup>* <sup>=</sup> 8.68, *<sup>p</sup>* <sup>=</sup> 0.007, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.27; **Figure 3**). Across trials 10 and 12, a main effect of condition was apparent with larger SCRs for the significant versus the indifferent condition (significant <sup>&</sup>gt; indifferent: *<sup>F</sup>* <sup>=</sup> 8.29 *<sup>p</sup>* <sup>=</sup> 0.008, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.26). Again, there was no trial by condition interaction, indicating that dishabituation did not differ between the two conditions.

#### **SKIN CONDUCTANCE LEVELS**

**Figure 4** demonstrates that across the 13 trials, pre-stimulus SCL was greater for the significant than the indifferent condition (significant <sup>&</sup>gt; indifferent: *<sup>F</sup>* <sup>=</sup> 10.06, *<sup>p</sup>* <sup>=</sup> 0.004, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.30).

Response decrement was evident, with SCLs over trials 1–10 demonstrating both linear and quadratic trends (linear trials:

*<sup>F</sup>* <sup>=</sup> 41.50, *<sup>p</sup>* <sup>&</sup>lt; 0.001, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.64; quadratic trials: *F* = 8.26, *<sup>p</sup>* <sup>=</sup> 0.009, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.26). There was a main effect of condition across trials 1–10, with larger SCLs for significant compared to indifferent stimuli (significant <sup>&</sup>gt; indifferent: *<sup>F</sup>* <sup>=</sup> 6.70 *<sup>p</sup>* <sup>=</sup> 0.016, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.23). **Figure 4** shows that SCLs reached a plateau from around trial 8 in the significant condition, and around trial 11 in the indifferent condition, but these trends did not statistically differ with condition.

**Figure 4** shows that sensitization is apparent as an increase in SCL from trial 11–12 (11 <sup>&</sup>lt; 12: *<sup>F</sup>* <sup>=</sup> 5.65, *<sup>p</sup>* <sup>=</sup> 0.026, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.20). SCLs were larger over both trials 11 and 12 for the significant than the indifferent condition (significant > indifferent: *<sup>F</sup>* <sup>=</sup> 9.25, *<sup>p</sup>* <sup>=</sup> 0.006, <sup>η</sup><sup>2</sup> <sup>p</sup>= 0.29). There was no trial by condition interaction.

#### **CORRELATIONS**

**Figure 5** shows the deviant sensitization, the increase in SCL from trial 11–12, as a function of deviant OR. In both conditions, this SCL increase (deviant sensitization) is correlated with the deviant OR: indifferent*r*(22) = 0.510, *p* = 0.005; significant*r*(22) = 0.424, *p* = 0.019, suggesting a similar origin (the novelty associated with the deviant). However, **Figure 6** demonstrates that this sensitization is not just a remnant of the incomplete resolution of the phasic response to the deviant. **Figure 6** shows the average electrodermal activity across all subjects for trials 11 and 12 separately for each condition. The vertical dashed lines represent stimulus onset (including the variable SOA at trial 12). The horizontal dotted lines illustrate that the phasic SCR was complete and had returned to pre-response levels within 10 s from trial 11 onset, and the diagonal dashed lines represent the continuation of this response. The shaded area above this is sensitization. It can be seen that the dishabituated response at trial 12 is not directly affected by the

**FIGURE 5 | Deviant sensitization (SCL: 12 minus 11) as a function of deviant ORs (SCR 11).** sensitization appears to be dependent on the deviant ORs for both conditions.

previous OR at trial 11, as that response has already resolved. This indicates that any increase in arousal from trial 11–12 is sensitization rather than a remnant of the preceding phasic response.

Dishabituation is shown as a function of sensitization in **Figure 7**. This shows that the increase in SCR from trial 10–12 (dishabituation) is independent of the increase in arousal for the same two trials, 10–12 (sensitized arousal), for both the indifferent *r*(22) = 0.210, *p* = 0.162 and the significant condition *r*(22) = 0.173, *p* = 0.209.

To test the prediction that the current arousal level determines OR magnitude, **Figure 8** shows SCR plotted against pre-stimulus SCL for each subject and trial for both conditions (24 × 13 × 2 = 624 data points). Over all trials

dependent on the current arousal level in both conditions.

and subjects, SCR magnitude was dependent on the current SCL for indifferent *r*(310) = 0.383, *p* < 0.001 and significant *r*(310) = 0.406, *p* < 0.001 stimuli, indicating that the current arousal level amplifies the phasic OR. There was no statistical difference between the two conditions *z* = 0.340, *p* = 0.367.

To explore the relationship between state and response as a function of trial, separate correlations were carried out between SCR and pre-stimulus SCL for each trial (separately for indifferent and significant). These 13 correlations had a mean for indifferent stimuli of *r* = 0.403, *SD* = 0.13 and for significant of *r* = 0.434, *SD* = 0.16; these did not differ significantly *z* = −0.120, *p* = 0.452. **Figure 9** shows the slope coefficients of the underlying scatterplots for these correlations plotted over trials for each condition. The relationship between the OR and the current arousal level (i.e., the amplification factor) apparently changes with stimulus novelty and significance. This was tested with a multiple regression using our three hypothesized determinants of the SCR. Novelty, significance, and arousal accounted for 55% of the variance in SCR, and the linear combination of these three variables significantly predicted SCR, *F*(3, 620) = 90.38, *p* < 0.001. The coefficients for novelty (β = 0.302), significance (β = 0.125), and arousal (β = 0.030) were all found to significantly contribute to SCR [novelty: *t*(623) = 9.64, *p* < 0.001, significance, *t*(623) = 6.37, *p* < 0.001, arousal *t*(623) = 10.88, *p* < 0.001].

#### **DISCUSSION**

The aim of this study was to determine the mechanism of dishabituation by testing separate predictions derived from dual process theory and Sokolov's neuronal-model of the OR. Our previous investigation, Steiner and Barry (2011), utilized a dishabituation

paradigm with a 5–7 s SOA, but this did not allow complete resolution of the phasic response preceding the dishabituation trial. In the current study, the extended SOA (13–15 s) allowed complete resolution of the response to the deviant trial, which served to demonstrate that sensitization, the change in arousal following the deviant, was genuine and not contaminated by the preceding deviant response. Dishabituation was not dependent on the change in arousal (sensitization)following the deviant, but instead, was dependent on the immediate pre-stimulus arousal level, as applies generally. This finding argues against dual process theory's explanation of dishabituation and provides support for Sokolov's assertion that dishabituation is a disruption to the habituation process.

As predicted, our phasic measure of the OR, the SCR, met the formal criteria for habituation (Thompson and Spencer, 1966; Rankin et al., 2009). That is, we observed *response decrement* over trials, *response recovery* to the interpolated deviant, and *dishabituation* to the representation of the habituated stimulus. The significance of the stimulus also affected OR magnitude, with larger SCRs for the significant compared to the indifferent condition. This is a robust finding that is consistent with previous research examining habituation of the electrodermal response (Barry, 2004, 2009; Steiner and Barry, 2011). Stimulus significance also affected the habituation of SCRs, with responses to indifferent stimuli showing a more systematic decline than SCRs to significant stimuli, a finding also in line with our previous investigations.

Our pre-stimulus state/arousal measure, SCL, decreased in magnitude over the first 10 trials and was greater over all trials for the significant compared to the indifferent condition, a finding consistent with Barry (2004). SCLs to significant stimuli did not continue to follow the same decremental pattern as the indifferent condition, and stayed somewhat elevated. It should be noted that the effects of stimulus significance (counting) observed here cannot be accounted for by Sokolovian or dual-process theories of the OR, as neither of these theories provide a mechanism for significance-related differences in tonic and phasic responding. Rather, an under-utilized theoretical concept, Maltzman's voluntary OR, predicts that both arousal level and the OR will be enhanced for motivationally significant stimuli.

Sensitization of arousal was observed following the deviant, with larger pre-stimulus SCLs for the post-deviant trial compared to the deviant trial. This finding replicates Steiner and Barry (2011) and is in line with Barry (2004) and Barry and Sokolov (1993), where the sensitization process was explored at the start of a habituation sequence. Here, sensitization, measured as the change in arousal from trial 11–12, was positively correlated with the deviant phasic OR at trial 11. Initially, this seems to suggest that sensitization is dependent on the deviant OR in both conditions, but as shown in **Figure 6**, sensitization is independent of the preceding deviant trial. Steiner and Barry (2011) also found this positive correlation, but only for the significant condition; this was due to the incomplete resolution of the deviant OR. In that study, the outcome of this test was used to justify restriction of the exploration of the mechanism of dishabituation to indifferent stimuli only. However, in the current study, we extended the SOA beyond the time needed for the phasic response to resolve, and, as **Figure 6**

shows, the OR to the deviant trial completely resolves in both conditions before the onset of the following stimulus. This suggests that sensitization is not directly dependent on the preceding OR to the deviant trial, but is rather an independent process occurring after this response has resolved.

Importantly, dishabituation was found to be independent of sensitization for both conditions, a finding extending Steiner and Barry (2011), where this was demonstrated for indifferent stimuli only. This argues against dual-process theory's unique assertion that dishabituation is nothing more than a superimposed process of sensitization, independent of habituation. Rather, it suggests that dishabituation actually reflects the increased novelty associated with the reinstatement of the H-process post-deviant. This provides support for Sokolov's assertion that dishabituation reflects a disruption to the habituation process.

We continued our investigation into the mechanism of dishabituation by exploring current arousal as Sokolov's responseamplifier. We tested this by correlating the OR with the prestimulus arousal level for each trial and each subject for both conditions. The phasic OR was found to be dependent on the current state, and this did not differ with condition. This finding confirms the importance of arousal as a response amplifier and is consistent with findings from continuous performance tasks (VaezMousavi et al., 2007a,b). To examine how this relationship changed over trials, OR was correlated with current arousal at each trial for both conditions. When *r*-values were averaged across trials, there was no statistical difference between the two conditions. However, the slope of the correlation scatterplots (amplification) appeared to change as a function of trial, differing between indifferent and significant conditions. This seems to reflect the totality that current arousal, novelty, and significance together determine phasic response magnitude. This was confirmed with a multiple regression, where all three determinants of the OR were found to predict the magnitude of SCR.

In sum, our findings show that the novelty and significance of a stimulus, and the current level of arousal, consistently predict the magnitude of the phasic OR, including the dishabituated response. Here, novelty reflected dual process theory's H-process, arousal was modeled on the S-process, and significance was based on Sokolov's description of significant stimuli. Sokolov did not provide a theoretical mechanism for the significance effects observed here, so this was examined in the context of Maltzman's voluntary OR. Importantly, we demonstrated that sensitization, the *change* in arousal, is a process that is separate from the resolution of the phasic response. Together, this suggests that dishabituation is a disruption of the habituation process, and the magnitude of this response is determined by the current arousal level.

The data presented here illustrate the process of response habituation and the influence of corresponding state changes in electrodermal activity. When examining slow-changing autonomic measures, such as the electrodermal response, the SOA should be long enough to ensure the complete resolution of responses. To further disentangle the phasic and tonic components of the OR, work integrating slower autonomic measures (e.g., electrodermal activity) with faster-changing central (e.g., EEG) measures of habituation and arousal is required. For example, using a dishabituation task, Barry et al. (2012) showed that an initial stimulus-induced transient increase in delta and theta EEG activity correlated with SCR, showing response decrement, recovery, and dishabituation. Future research should seek to confirm the central-neural mechanism of dishabituation. In that case, dishabituation of central measures, such as components in the event-related potential of the brain, should reflect a change in cortical excitation, demonstrable with EEG measures. Current work in our laboratory is exploring this.

# **REFERENCES**


VaezMousavi, S. M., Barry, R. J., Rushby, J. A., and Clarke, A. R. (2007a). Evidence for differentiation of arousal and activation in normal adults. *Acta Neurobiol. Exp.* 67, 179–186.

VaezMousavi, S. M., Barry, R. J., Rushby, J. A., and Clarke, A. R. (2007b). Arousal and activation effects on physiological and behavioral responding during a continuous performance task. *Acta Neurobiol. Exp. 67*, 461–470. doi: 10.1027/0269-8803. 19.2.91

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 11 September 2013; paper pending published: 28 November 2013; accepted: 23 January 2014; published online: 14 February 2014.*

*Citation: Steiner GZ and Barry RJ (2014) The mechanism of dishabituation. Front. Integr. Neurosci. 8:14. doi: 10.3389/fnint.2014.00014*

*This article was submitted to the journal Frontiers in Integrative Neuroscience.*

*Copyright © 2014 Steiner and Barry. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Habituation, sensitization, and Pavlovian conditioning

# *Münire Özlem Çevik\**

*Psikoloji Bolumu, TOBB Ekonomi ve Teknoloji Universitesi, Ankara, Turkey*

#### *Edited by:*

*Catharine H. Rankin, University of British Columbia, Canada*

#### *Reviewed by:*

*David L. Glanzman, University of California Los Angeles, USA Jean-René Martin, CNRS, France*

*\*Correspondence:*

*Münire Özlem Çevik, Psikoloji Bolumu, TOBB Ekonomi ve Teknoloji Universitesi, Sogutozu 06520, Ankara, Turkey e-mail: mcevik@etu.edu.tr*

In this brief review, I argue that the impact of a stimulus on behavioral control increase as the distance of the stimulus to the body decreases. Habituation, i.e., decrement in response intensity repetition of the triggering stimulus, is the default state for sensory processing, and the likelihood of habituation is higher for distal stimuli. Sensitization, i.e., increment in response intensity upon stimulus repetition, occurs in a state dependent manner for proximal stimuli that make direct contact with the body. In Pavlovian conditioning paradigms, the unconditioned stimulus (US) is always a more proximal stimulus than the conditioned stimulus (CS). The mechanisms of associative and non-associative learning are not independent. CS−US pairings lead to formation of associations if sensitizing modulation from a proximal US prevents the habituation for a distal anticipatory CS.

#### **Keywords: proboscis extension response, habituation,** *Drosophila melanogaster***, olfactory conditioning, mushroom bodies, dopamine**

Identification of the necessary and sufficient conditions for the formation of associations has been a driving influence on learning theory and research. In Pavlovian conditioning, a conditioned stimulus (CS) acquires the ability to trigger a new response by virtue of being paired with an unconditioned stimulus (US), which by definition is biologically important and capable of triggering an innate reflex. Starting with British associationism, early theories of conditioning were based on the premise that temporal contiguity was both necessary and sufficient for stimulus associations (Gormezano and Kehoe, 1981). Although the temporal coincidence of the CS−US pair is still accepted to be necessary, research since late 1960's presented irrevocable challenges to its sufficiency for the formation of associations (Durlach, 1989). Of particular importance was the discovery of blocking (Kamin, 1968), where an association fails to be formed in spite of the seamless temporal contiguity between the CS and the US, if the CS is presented in a compound with another CS that had previously been associated with the same US. Blocking had a major bearing on the development of the contingency theory of associative learning (Rescorla and Wagner, 1972), which has been a major breakthrough that diverted the focus of learning research from physical properties (e.g., intensity and temporal coincidence) to the signal value [e.g., predictiveness, informativeness, (un)expectedness] of the to-be-associated stimuli (Rescorla and Holland, 1988; Gallistel and Matzel, 2013).

The signal value of a stimulus is correlated with its potential to support responsiveness, which for a given set of physical parameters depends largely on the history of non-associative learning, i.e., habituation and sensitization for the stimulus. Habituation refers to the reduction in the probability or amplitude of responding that is observed upon inconsequential stimulus repetition. For example, repeated delivery of an odor at constant inter-stimulusintervals (ISI) would eventually lead to the habituation of the response that is initially triggered by the odor. I say eventually, because depending on the parameters of the stimulation protocol (e.g., odor concentration, frequency of odor presentation), a temporary increment in responsiveness might initially be observed. If, however, an appetitive gustatory stimulus (e.g., sugar) is repeated with the same ISI, depending again on the concentration of sugar, frequency of stimulation, and the physiological state of the organism, this protocol is likely to result in an increment in the probability of responding, i.e., sensitization. Finally if odor and sugar are paired instead of being presented separately, the standard paradigm for Pavlovian conditioning would ensue, where the two stimuli would now be termed conditioned (CS) and unconditioned stimuli (US), respectively. Hence, associative learning can be suggested to entail an evasion from habituation for the CS as it signals the arrival of a sensitizing event, the US, and a conditioned response (CR) would then be triggered during the CS in anticipation of the US.

It follows that whether or not coincident pairings of the same CS and US will yield associative learning is influenced by the signal value, or equivalently, on the history of habituation and sensitization prior to conditioning. If, for example, a CS is repeatedly presented to yield habituation prior to conditioning, then its signal value would be reduced, which would in turn reduce subsequent associative learning relative to a control condition where the CS is presented *de novo* during Pavlovian pairings. Indeed, the strength of associations decreases if organisms are pre-exposed to the CS before conditioning (Reiss and Wagner, 1972; Hall, 2001). Similarly, efficacy of a US can be potentiated if conditioning is preceded by a sensitizing treatment (LoLordo and Randich, 1981), and vice versa (see Pearce and Hall, 1980; Franklin and Hall, 2011, for explanations based on context-conditioning during US preexposure). In the same vein, the failure of a US to support new associations under a blocking paradigm can be correlated with the reduction in sensitization (or the "surprise value" of the Rescorla-Wagner model) evoked by a signaled (as opposed to an unexpected) US, and indeed, US efficacy is known to decrease with extended associative training (Rescorla and Wagner, 1972).

The suggestion that conditioning emerges as a sensitizing stimulus (US) exerts a modulatory influence to prevent the habituation for another stimulus (CS) begs the identification of stimulus characteristics that determine whether habituation or sensitization will occur upon repetition. So, are there any independent criteria for predicting *a priori* if a stimulus can incite sensitization or habituation, or equivalently, are there any inherent properties that assign a CS− or US-like function to a stimulus?

## **THE IMPACT OF A STIMULUS DECREASES WITH ITS DISTANCE FROM THE BODY**

Only a small number of model paradigms for Pavlovian conditioning endured the test of time in vertebrate and invertebrate species. The limiting factor is the scarcity of stimuli that can function as a US. In all paradigms that yield reliable conditioning, the US is a proximal stimulus that comes into direct contact with the body. For example, in appetitive conditioning paradigms, food is the most frequently used US which supports conditioned approach or discrimination via its gustatory or nutritive properties. In aversive conditioning, the US is most often a pain inflicting stimulus (e.g., heat or electric shock) that is delivered via the tactile domain. In contrast, the CS's consist of distal inputs from the visual, auditory, or olfactory domains. Distal stimuli are conspicuously ineffective as US's even after they have acquired the potential to support conditioned responding through higherorder conditioning (Rescorla, 1973). In fact, there does not exist a robust model paradigm which utilizes a distal stimulus as the US, and a proximal stimulus as the CS. The more proximal nature of the US relative to the CS is true even for paradigms where the CS and the US are first encountered through the same sensory modality. For example, in taste preference conditioning, the gustatory stimulus is the CS, and the nutritive value of food which requires further digestive processing, is the US (de Araujo, 2011; Fujita and Tanimura, 2011). Similarly, digestion-related malaise functions as the US in conditioned taste aversion (Garcia and Koelling, 1966).

The distinction between distal and proximal stimuli is nontrivial because it matches the temporal order of events as the animal moves and/or the environment changes. For example, one can smell an apple before/without tasting it, but not vice versa. In general, biologically important objects (e.g., food, predators, aggressive opponents, potential mates, sharp objects) are detectable via distal cues (i.e., odors, sounds, sight) before they contact the body to activate proximal senses (taste, touch), and associative learning is sensitive to this temporal order (Timberlake, 1994). According to the argument being raised here, associative learning will occur selectively for distal signals that provide anticipatory cues to biologically important stimuli, and habituation will follow otherwise. Finally, it should be noted that by the very definition of Pavlovian conditioning, this argument applies to distal stimuli that are neutral prior to conditioning, i.e., stimuli that do not by themselves evoke an appetitive or aversive response, but acquire the ability to do so after being paired with a US.

#### **MODULATION OF DISTAL STIMULUS PROCESSING BY PROXIMAL INPUTS AS A GENERAL PRINCIPLE FOR THE ORGANIZATION OF LEARNING AND BEHAVIOR**

All behaviors can be coarsely classified into approach and avoidance where the animals choose to respond in a manner to maintain or terminate the impact of impinging stimuli. As classical ethologists of the last century astutely documented (Tinbergen, 1951), approach-based behavior systems (e.g., foraging and mating where the animals' movements are directed toward appetitive stimuli, and aggression where the animal approaches aversive stimuli) have two properties in common: The first is the sequential pattern of behavior which requires persistent sign tracking and stimulus evaluation, presenting the animal with multiple check points before it finally commits the consumatory act. The second is both state- and stimulus-dependence of performance, which attunes behavioral choice to physiological demands.

Often referred to as motivation, the impact of internal state variables on behavior is robust to fluctuations in environmental inputs, maintaining the coherence and goaldirectedness of behavior in the face of environmental perturbations. Interestingly, when animals are in a conflict between acting in accordance with the existing motivational state or responding to an incompatible external stimulus, their choice reflects a high impact of internal state variables. Similarly, proximal stimuli are more likely to control behavior when they are in conflict with distal stimuli. For example, initiation of swimming in response to noxious mechanical stimulation is suppressed in hungry, but not sated leech (Gaudry and Kristan, 2012). Similarly, male fruit flies fail to escape from mechanical turbulence or dangerously high ambient temperatures when they are mating (Crickmore and Vosshall, 2013). Other examples abound in nature: The type and intensity of the behavioral response to a conspecific is highly modulated both by the motivational state of the animal, and the distance of the intruder. These observations suggest a body-centered hierarchy of impact in the decreasing order of internal state, proximal, and distal stimuli, where closeness to the body yields not only a higher probability of access to behavioral control, but also the potential to modulate the effects of more distal input sources.

One would only expect that the organization of learning mirrors the organization of behavior. Experience dependent change in the feeding reflexes of fruit flies provides a good example. Flies respond to appetitive stimulation of their gustatory receptors by extending their mouthparts, the proboscis. In the presence of a tastant, elicitation of the proboscis extension reflex (PER) is a probabilistic process that is modulated by stimulus properties (e.g., type and concentration of the appetitive solution Dahanukar et al., 2007), physiological state (e.g., hunger Inagaki et al., 2012; Marella et al., 2012, nutrition Edgecomb et al., 1987, and arousal Dethier, 1974; Vargo and Hirsch, 1982), and memory (Chabaud et al., 2006). For example, habituation of PER is observed upon repetitive stimulation of the tarsal receptors with sucrose in the fruit fly *Drosophila melanogaster*. When flies are re-habituated an hour later, PER probability decreases faster relative to the *de novo* control group that is equated for the level of initial sucrose responsiveness, indicating that habituation memory for PER may last for at least an hour (Cevik and Erden, 2012).

Markedly, expression of both short-term habituation and 1-h habituation memory are strictly dependent on the physiological state of the flies. In the above study (Cevik and Erden, 2012), individual flies exhibited one of three distinct response patterns under a PER habituation protocol following 1–4 h of food deprivation: Some flies were completely non-responsive to sucrose irrespective of its concentration. Another group of flies exhibited the converse response pattern and failed to habituate, and again, their responsiveness did not change with sucrose concentration. Finally, a third group of flies responded to sucrose, and habituated upon repeated stimulation.

Two points are worthy of special emphasis regarding the above data set on PER habituation: First and foremost, the effects of stimulus- and state-dependent factors on sucrose responsiveness are clearly non-additive. That is, stimulus factors are not equivalent with respect to their impact on controlling PER or its experience-dependent modulation when compared to the internal state. In fact, the physiological state determined whether and how the flies would respond to appetitive stimuli: Following 1 h of food deprivation, 60% of flies were completely non-responsive to 600 mM sucrose, as if it was not there. At 4 h of food deprivation, less than 20% of the flies remained non-responsive, whereas 40% of the flies failed to show habituation, and continued to respond as if there was nothing else but sucrose. Clearly, state-dependence of responsiveness had a qualitative, but not quantitative effect on how appetitive stimuli were processed for immediate responding as well as for memory-formation.

Second, habituation, when it was observed, was rapid. It can be argued that stimuli exert the highest influence on behavior when they are novel. If a fly emitted at least one PER during the habituation session, its first response was highly likely to occur within the first two trials. To put it in other words, if a fly did not respond in the first few instances of sucrose presentation, it was not likely to respond afterwards. Because the flies are not permitted to ingest sucrose or even touch the sucrose solution with their proboscis in a habituation session, PER emissions do not result in feeding. Under these conditions, tarsal stimulation with sucrose is redundant, and entails rapid habituation. Notice that although neural adaptation and habituation are not dissociable, habituation cannot be reduced to adaptation either. For example, in the above experiment, the same stimulus repetition protocol which supposedly produced similar rates of input adaptation, led to different rates of habituation as a result of hunger modulation. Further, both habituation and its failure could be predicted in advance by early response parameters that were not confounded by adaptation. For example, responsiveness within the first two trials could predict the subsequent pattern of habituation before adaptation was in effect. In general, although short-term habituation is correlated with failures of sensory transmission (Malkinson and Spira, 2013), longer term habituation cannot be reduced to either receptor adaptation or a depression of the sensorimotor synapses, presenting *prima facie* evidence for experience dependent change in the likelihood of responding to the same stimulus (Glanzman, 2009).

In fact, habituation might be the default fate for most stimuli in the absence of top-down modulation (although see Horn and Hinde, 1970 for examples of non-habituating reflexes). To state it in more general terms, habituation is not just a waning of responsiveness for repetitive external stimuli, but is a default property of central processing unless it is modulated by a sensitizing input. This framework might be useful in understanding organization of behavior and learning. Below, I present a brief review of recent findings on appetitive olfactory conditioning in fruit flies to suggest that classical conditioning ensues when the habituation of a distal stimulus is prevented by virtue of its anticipatory pairing with the proximal, sensitizing stimulus under the permissive context of the internal milieu.

# **APPETITIVE OLFACTORY CONDITIONING AS A MODEL TO STUDY CROSS-MODAL STIMULUS INTEGRATION**

In an appetitive olfactory conditioning paradigm, a group of flies are first exposed to an odor, CS+, which is simultaneously presented with sucrose, US. In alternating trials, they are also exposed to another odor, CS−, which is not accompanied by an appetitive US. During the test phase, the flies are simultaneously presented with CS+ and CS− in the absence of a US, where they choose to approach CS+ following successful conditioning. Notice that the training protocol equates the two olfactory stimuli, CS+ and CS−, with respect to habituation while differentiating them in terms of the sucrose-driven sensitization that follows (Tully and Quinn, 1985).

Mushroom bodies (MB) are bilateral multi-modal sensory integration sites (Strausfeld et al., 2003) that have been associated with appetitive and aversive memories (Mizunami et al., 1998; Keene and Waddell, 2007; Zhang et al., 2013), attention (van Swinderen, 2007), and context-(Liu et al., 1999) and saliencebased decision making (Zhang et al., 2007) in several insect species. The MB in each hemisphere of the adult fruit fly brain house ∼2500 Kenyon cells which form lobular structures that can be distinguished in terms of their morphology and function. For example, the axons of a subset of Kenyon cells bifurcate to form the vertical α and medial β lobes, another subset likewise forms the α and β lobes, and γ neurons have unbranched axons that project medially. Kenyon cell dendrites are housed in the calyces where they receive olfactory input from the projection neurons that ascend from the antennal lobes. In turn, Kenyon cell output converges on a small number (∼30 pairs) of extrinsic neurons that innervate distinct areas along MB lobes (Chen et al., 2012; Pai et al., 2013; Placais et al., 2013). Finally, at both the calycal input and the lobular output areas, Kenyon cells make extensive preand post-synaptic contacts with aminergic and peptidergic modulatory neurons that relay the computed impact of proximal (i.e., gustatory and tactile) input as well as the internal state (Krashes et al., 2009; Mao and Davis, 2009; Pitman et al., 2011; Aso et al., 2012; Burke et al., 2012).

The innervation pattern of modulatory neurons shows extensive overlap with those of the extrinsic output neurons, defining lobular areas that are functionally specialized for different types and/or stages of memory processing (Krashes et al., 2007; Aso et al., 2012; Xie et al., 2013). For example, a broad, non-selective activation of the α β lobes is observed immediately following the training phase of olfactory conditioning. This activity overlaps in space and time with the activity of octopaminergic (OA) neurons that are incited by the sweet taste of the sugar (US), which can modulate conditioned odor preference for as short as a few minute. Longer lasting memories require the activity of dopaminergic (DA) neurons in the PAM cluster (Burke et al., 2012; Liu et al., 2012). A subset of PAM-DA neurons receive input from both the OA neurons that relay the impact of sweetness, and a yet unidentified source that relays information about the nutritive quality of sugar shortly after ingestion. Being the convergence point of the sensory intensity and nutritive quality of food, DA neurons of the PAM cluster in turn innervate the tip of the β and γ lobes to support olfactory memories that may last for hours (Burke et al., 2012).

The sequential pattern of OA and DA modulation of the Kenyon cells which is required for the formation of appetitive memories can be interpreted as a selection-for-not-habituation of olfactory representations as they are modulated by sensitizing input from progressively more proximal sources. For example, sweetness is a proximal (yet external) input that provides information about the concentration of food, and it can facilitate the associability of mushroom body olfactory representations for minutes via OA modulation. More stable memories require sensitizing modulation from an internal source: DA neurons that relay the impact of the nutritive quality of food can support olfactory memories for hours. Hence, following conditioning, when animals encounter distal olfactory stimuli, they learn to anticipate the sequential arrival of progressively more proximal inputs with positive valence, namely sweetness and nutritive value, through the DA-mediated olfactory associations (see Wittmann et al., 2005; Tully et al., 2007; Howe et al., 2013 for similar examples in the vertebrate brain). Converging evidence comes from a study which showed that in addition to their well-known roles in associative learning, MBs are also involved in sustaining the impact of biologically important stimuli. The disruption of MB function results in a premature habituation to electric shocks that can otherwise function as US's to support aversive olfactory associations (Acevedo et al., 2007).

A similar argument can hold for the retrieval of appetitive olfactory memories. Memory retrieval is inferred as the animal performs the CR; so by definition, it is a process that guides action selection. Therefore, one can imagine that upon encountering the CS, the process of memory retrieval initiates dynamics that bias responding in favor of US anticipation or consummation. In the fly brain, the αβ lobes have been shown to be required for the retrieval of short- and long-term memories (Aso et al., 2012; Xie et al., 2013), so arguably, these lobes are involved in accessing associations of olfactory stimuli to guide anticipatory action selection, irrespective of the temporal phase or stability of memory traces. In accordance with the model being proposed here, it is only expected then, that the αβ lobes would be modulated by the impact of proximal and internal inputs. Interestingly, a zone located at the heel of MB's where α and β lobes intersect, acts as a switch that biases memory-driven performance in favor of approach or avoidance responses. This zone is innervated by a subset of DA neurons of the PPL1 cluster, MB-MP1 neurons, activation of which is necessary for the retrieval of aversive olfactory memories, i.e., for the selection of actions that are compatible with moving away from the odor source (Mao and Davis, 2009; Xie et al., 2013). However, these MB-MP1 neurons can be inhibited by inputs that convey the impact of proximal appetitive substances as well as the internal milieu that bias action selection in favor of approach. For example, information about the sweetness of the tastant [via the OA neurons (Burke et al., 2012)] and the physiological state of the fly [via neuropeptide F (Krashes et al., 2009)] converge to inhibit the activity of a subset of PPL1-DA neurons that convey negative valence. This pattern of connectivity reduces the probability of moving away from high-quality tastants and increases the probability of approaching a food source using learned associations, when the animal is hungry.

Notice that the hierarchical modulation of olfactory representations in downstream multi-modal association areas confers flexibility and context-dependence to conditioned responding (Strausfeld, 2012). Being driven both by the internal and proximal environment of the animal, DA neurons modulate the processing of distal stimuli to confer context-dependent salience to a selected subset (i.e., attention), endure the impact of previously inconspicuous distal stimuli by associating them with significant proximal events (i.e., conditioning and memory formation), and mediate the selection of a general action plan *vis a vis* distal stimuli (i.e., approach vs. avoidance).

Finally, it should be noted that the current model suggests a resemblance between the properties of neural circuits that underlie associative learning in vertebrate and invertebrate brains. NMDA-receptor mediated long term potentiation (LTP) has been the principal model of memory formation in the mammalian brain since its discovery by Bliss and Lomo (1973). In the excitatory synapses that undergo LTP, a weak input can become potentiated as a consequence of being coincident with a stronger input (e.g., Barrionuevo and Brown, 1983); and this associative property has validated LTP as a cellular model of Pavlovian conditioning. Although the role of monoamines in environmentspecific drug effects and reinforcement learning have long been established as models of conditioned responding in the mammalian brain (e.g., Stewart and Vezina, 1988), monoaminergic modulation of LTP induction has caught researchers' interest more recently. For example, monoaminergic enhancement of hippocampus-dependent memory formation is observed when the emotional valence of the US is relevant (Wittmann et al., 2005), and monoamine transmission is often heterosynaptically activated by the basolateral nucleus of the amygdala (BLA) (Tully et al., 2007). Interestingly, bilateral lesions of BLA produce symptoms that resemble the Kluver-Bucey syndrome (Davis and Whalen, 2001). This condition was originally described when the bilateral removal of temporal lobes including the amygdala caused monkeys to compulsively attend to almost every visual stimulus and proceed to examine even repulsive objects by mouth, increase heterosexual and homosexual behavior, and approach conspecifics as well as human caretakers with a marked absence of anger or fear. In essence, these symptoms reveal a failure to habituate to distal inputs, which results in an inappropriate progression toward initiating and maintaining proximal contact with stimuli irrespective of their incentive value. Similarly, brains.

bilateral lesions of BLA (Hatfield et al., 1992), or the depletion of DA and NE in the amygdala (Fernandez-Ruiz et al., 1993) produces a decrement in taste-potentiated odor aversion, but not in taste-aversion *per se*. These results suggest that the monoamine modulation in the BLA functions to guide conditioned avoidance reactions toward distal (e.g., olfactory) stimuli by the signaling the valence of proximal inputs (e.g., taste). Interestingly, olfactory conditioning in fruit flies has recently been shown to involve postsynaptic plasticity in the dendrites of MB output neurons that express NMDA receptors (Xia et al., 2005; Chen et al., 2012; Pai et al., 2013), which closely overlap with the arborizations of DA neurons along the lobes. It can then be argued, that input from proximal receptors initiates monoamine signaling, which in turn modulates excitatory synapses to establish conditioned approach

or avoidance of distal stimuli in both vertebrate and invertebrate

# **SUMMARY AND CONCLUSION**

In this brief review, I argued that the impact of a stimulus on behavior and its potential to modulate the effects of other stimuli increase as its distance from the body decreases, and the bodycentered hierarchy of stimulus impact applies to the organization of behavior as well as its experience dependent change. For example, the likelihood of habituation, i.e., the inability of a perceivable stimulus to control behavior is higher for distal stimuli, whereas the likelihood of sensitization is higher for stimuli that come into direct contact with the body to activate gustatory and/or mechanoreceptors. Further, I argued that mechanisms of associative learning are not independent from those of habituation and sensitization. Pavlovian conditioning ensues when internal state (e.g., hunger) up-regulates the sensitizing potential of the proximal US (e.g., food), which in turn prevents habituation to the distal CS (e.g., odor). According to the argument being raised here, CS−US pairings should fail to yield associative learning if sensitization does not occur to the US. In this short review, I gave examples from olfactory learning in fruit flies, but I believe that a similar hierarchy of body-centered stimulus impact exists in the vertebrate brain as well. The idea that sensitization from proximal inputs and the internal milieu drives associative learning might help our understanding of phenomena such as the development of chronic pain (Agroff et al., 2009), or eating disorders (Kaye et al., 2011) as failures of habituation.

#### **REFERENCES**


TANGO-mapping of dopamine signaling reveals appetite control of sugar sensing. *Cell* 148, 583–595. doi: 10.1016/j.cell.2011.12.022


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 31 August 2013; accepted: 23 January 2014; published online: 11 February 2014.*

*Citation: Çevik MÖ (2014) Habituation, sensitization, and Pavlovian conditioning. Front. Integr. Neurosci. 8:13. doi: 10.3389/fnint.2014.00013*

*This article was submitted to the journal Frontiers in Integrative Neuroscience.*

*Copyright © 2014 Çevik. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*