Department of Neurobiology and Anatomy, Wake Forest University School of Medicine, Winston-Salem, NC, USA
Sensory processing disorder (SPD) is characterized by anomalous reactions to, and integration of, sensory cues. Although the underlying etiology of SPD is unknown, one brain region likely to reflect these sensory and behavioral anomalies is the superior colliculus (SC), a structure involved in the synthesis of information from multiple sensory modalities and the control of overt orientation responses. In the present review we describe normal functional properties of this structure, the manner in which its individual neurons integrate cues from different senses, and the overt SC-mediated behaviors that are believed to manifest this “multisensory integration.” Of particular interest here is how SC neurons develop their capacity to engage in multisensory integration during early postnatal life as a consequence of early sensory experience, and the intimate communication between cortex and the midbrain that makes this developmental process possible.
One of the primary manifestations of sensory processing disorder (SPD) is an abnormal response to sensory cues that often manifests as hypo-responsiveness or hyper-responsiveness. Especially problematic is the inability of children with SPD to properly deal with cues from a variety of senses in order to initiate and control appropriate behaviors. Jean Ayres, a pioneer in this field, first referred to this disorder as Sensory Integration Disorder. She believed that it was due to a disruption in the brain’s use of sensory information to organize appropriate responses during its development and maturation (see Bundy et al., 2002
; Schaaf and Miller, 2005
). However, it is unclear what particular level or levels of the neuraxis are affected by this disorder, requiring investigators to account for all possibilities. The many other contributions to this Special Issue will deal more extensively with what is known about these issues. It is, however, important to note that sensory integration problems similar to those found in SPD have also been noted in other developmental disorders, such as autism, wherein individuals often experience abnormalities in multisensory integration by focusing extensively on visual processes and neglecting auditory signals (Lovaas et al., 1974
; Berkell et al., 1996
), and also in dyslexia and attention deficit disorder (Hairston et al., 2005
; Tucha et al., 2006
; Blau et al., 2009
).
Regardless of the etiology of SPD or the particular form it takes in a given individual, what appears to be fundamentally altered in the brain is its sensory processing and sensorimotor transduction. The identification of specific genetic and/or neurological processes that underlie SPD have thus far eluded clinicians and scientists; however, one midbrain circuit involving the superior colliculus (SC) may provide a working model to better understanding manifestations of its underlying dysfunction. This sensorimotor structure is involved in integrating sensory information and in initiating and controlling the motor responses that are involved in attentive, localization, and orientation behaviors (see Stein and Meredith, 1993
). Its functional integrity is very much dependent on a balance of inputs from different levels of the neuraxis, but is especially sensitive to inputs from association cortex (Stein, 2005
). The maturation of its sensory properties takes place postnatally as a result of interactions with the environment (Wallace and Stein, 2007
), and its function is to determine the immediate behavioral responses to the onset or change of sensory cues (see Stein and Meredith, 1993
).
Of particular concern here is to understand the postnatal process through which SC neurons develop their properties and refine their ability to use multiple senses synergistically in the process of transforming incoming sensory information into outgoing motor commands. Understanding the postnatal elaboration of sensory and/or motor circuits may provide insights into how disruptions that take place at different maturational time points can have different consequences and may result in different neural strategies for dealing with complexes of sensory stimuli. This approach may help explain some of the striking variations in the dysfunction profiles of patients with early disorders in the processing of sensory information (see Miller et al., 2001
). Using an animal model, the discussion below will focus on the developmental antecedents necessary for populations of SC neurons to develop their capabilities to integrate cues from different senses in order to initiate adaptive behaviors.
The brain has evolved to adapt information acquisition and response systems to produce rapid and accurate reactions to immediate environmental challenges. In order to facilitate these behaviors it has also created storage and retrieval systems that modify response strategies based on experience. The success in the evolving brain’s ability to use information quickly and efficiently was determined by whether any particular prototype would become a platform for further modification or would simply be discarded along with its owner. Nature is ruthless in its evaluation of innovation.
One result of these evolutionary pressures is that all brains utilize multiple sensory systems to extract information from different forms of environmental energy. By being tuned to different energy sources, each of these systems reflects a slightly different perspective of initiating environmental events. This is of particular benefit because those events or objects that cannot readily be identified along a single sensory dimension (e.g., how they look) may be identified along another (how they sound or feel) or through a combination of sensory dimensions. Combining sensory inputs is an extremely efficient technique for maximizing information utilization, and the brain employs a simple logic in using this multisensory information synergistically (Stein and Meredith, 1993
). Indeed, the brain appears to synthesize this information in an optimal fashion (e.g., Ernst and Banks, 2002
; see also Calvert et al., 2004
for general discussions). This process is called “multisensory integration,” one that enhances the probability of detecting the source of a signal, correctly identifying it, and, most importantly in the present context, properly reacting to it. Using its sensory machinery in this way, the brain achieves performance levels of effectiveness and efficiency that would be impossible if it were only able to use the senses independently.
But the brain is not automatically endowed with the full capability to use its multiple sensory systems in concert. Indeed, it uses them independently during early neonatal stages of development. Although it might be capable of matching information from one to another very early in life, it cannot synthesize their information to produce a new “integrated” multisensory response (see the discussion below). How it acquires this remarkable ability during postnatal life, and the physiological and behavioral consequences that can occur when this developmental process is disrupted, will be the primary subjects of this discussion. But first it is necessary to examine the normal operational features of multisensory integration, the underlying neural circuitry that is essential for its functional integrity, and the behavioral impact of this neural process.
Multisensory integration works by significantly modulating the physiological salience of information in the brain (Stein et al., 2004
), and its impact on perception and behavior is well-known (Talsma et al., 2006a,
b
, 2007
; King and Palmer, 1985
; Stein et al., 1989
; Hughes et al., 1994
; Frens and Van Opstal, 1995
; Corneil and Munoz, 1996
; Wallace et al., 1996
; Liotti et al., 1998
; Recanzone, 1998
; Zangaladze et al., 1999
; Grant et al., 2000
; Sathian, 2000
, 2005
; Marks, 2004
; Newell, 2004
; Sathian and Prather, 2004
; Schroeder and Foxe, 2004
; Shams et al., 2004
; Woldorff et al., 2004
; Woods and Recanzone, 2004
; Busse et al., 2005
; Weisser et al., 2005
; Ghazanfar and Schroeder, 2006
; Lakatos et al., 2007
). It has been shown to aid in disambiguating events, including those involving human speech and animal communication (Sumby and Pollack, 1954
; Bernstein et al., 2004
; Massaro, 2004
; Partan, 2004
; Ghazanfar et al., 2005
; Sugihara et al., 2006
), and to enhance and speed the detection, identification and reaction to external events (Stein et al., 1989
; Stein and Meredith, 1993
; Hughes et al., 1994
; Frens and Van Opstal, 1995
; Corneil and Munoz, 1996
; Marks, 2004
; Newell, 2004
; Sathian and Prather, 2004
; Shams et al., 2004
; Woods and Recanzone, 2004
). The survival benefits of such a process are obvious.
The general “rule of thumb” of how multisensory integration operates is as follows: signals from the different individual senses that are likely to be relevant to the same event are normally concordant in space and time. As such, these spatiotemporally aligned cross-modal stimuli produce markedly enhanced signals in the adult brain. These enhanced signals lead to a higher probability of detecting and reacting to the event. Cues from the different senses that are neither in temporal nor spatial register are likely to be unrelated and will fail to produce such enhancement. Indeed, under some circumstances, such cues might be treated as competitors, leading to mutual signal degradation and decreased likelihood of detecting and reacting to either cue.
Multisensory integration enriches our sensory experiences and increases the accuracy of our judgments of environmental events. It is interesting to note that the potency of combining cues from different senses increases as their individual effectiveness decreases, referred to as “inverse effectiveness” (see Stein and Meredith, 1993
). Thus, multisensory integration is of maximal utility when task difficulty increases, either because stimuli are weak or background noise is strong. This issue is of particular relevance to those suffering with SPD.
Although all brains engage this strategy of sensory processing and do so at multiple levels of the neuraxis (Stein and Stanford, 2008
), surprisingly little attention has been directed toward understanding how it develops in the young brain and/or the inherent plasticity in this system. Newer studies have shown that its operational principles are very sensitive to experience and that there is considerable postnatal development in these processes. Experiments in animals strongly suggest that multisensory integration is acquired postnatally and only after considerable sensory experience (Wallace et al., 1993
, 2006
; Wallace and Stein, 1997
, 2007
). As noted earlier, this capacity refers to the ability to synthesize information from different senses so that there is a new product. Very recent studies in human subjects seem to agree with this developmental profile by showing a gradual postnatal elaboration of multisensory integration capabilities (see Bremmer et al., 2008a
,b
; Neil et al., 2006
; Putzar et al., 2007
; Gori et al., 2008
). This should not be taken to mean that the maturation of every brain structure with multisensory neurons has already been explored, or that no communication across the senses is possible at birth. Some multisensory processes, like cross-modal matching and the recognition of amodal stimulus properties (e.g., size, intensity, frequency) have been demonstrated in very young children and infants (e.g., see Streri and Gentaz, 2003)
, and some infer that they are already present to some degree at birth (e.g., for discussions see Bremmer et al., 2008a
,b
; Flom and Bahrick, 2007
; Gibson, 1966
, 1969
, 1979
; Werner, 1973
; Bower, 1974
; Marks, 1978
; Meltzoff and Borton, 1979
; Meltzoff, 1990
; Morrongiello et al., 2003
; Streri and Gentaz, 2003
; Lewkowicz and Kraebel, 2004
; Lickliter and Bahrick, 2004
; ; but see also Maurer et al., 1999
). However, the technical difficulty in evaluating these capabilities before the neonate is able to obtain substantial experience with cross-modal cues is obvious, and this issue also deserves continued exploration.
As noted earlier, the SC is a midbrain structure that plays a significant role in attentive, localization and orientation behavior. A sudden event will engage its circuitry so that the event is detected, located in space and responded to with a shift of the eyes and head, and sometimes the entire body. The result is that the individual faces the source of that input and is in the best position to further evaluate and react to it. The SC is a layered structure and its deeper layers contain neurons responsive to visual, auditory or somatosensory (tactile) stimuli, and oftentimes to more than one sensory modality. This structure is an excellent model system for exploring the maturation of multisensory integration because of the volume of information about the maturation of its unisensory properties (Stein, 1984
), because it is a primary site of sensory convergence (Stein and Meredith, 1993
; Wallace et al., 1993
), because of the extensive information already available about its multisensory processes in the adult (see Stein and Stanford, 2008
for a recent review), and because much of what is already known about the development and organization of multisensory integration comes from this model (Stein et al., 1973a
, 1976
; Stein and Arigbede, 1972
; Stein and Clamann, 1981
; Stein, 1984
; Peck, 1987
; Stein and Meredith, 1993
; Stein et al., 1993
; Wallace, 2004
) but see also (Jay and Sparks, 1987a
,b
; Groh and Sparks, 1996a,
b
; Zwiers et al., 2003
; Barth and Brett-Green, 2004
; Calvert and Lewis, 2004
; Gutfreund and Knudsen, 2004
; King et al., 2004
; Sathian and Prather, 2004
; Woods and Recanzone, 2004
; Lakatos et al., 2007
; Senkowski et al., 2007
). In addition, the fact that it has a well-defined role in overt behavior makes it possible to examine the behavioral implications of its physiological properties.
Multisensory neurons have been studied most extensively in the cat, and it is often used as a source of information about multisensory integration at the level of the single neuron. It will thus be the primary model of reference below; however, relevant information from the monkey and other species will be incorporated to illustrate the generality of the observations. The functional circuits underlying SC multisensory integration and its maturation are complex, and the simple convergence of afferents onto a given neuron does not guarantee it will develop the ability to integrate cross-modal information, nor does it specify how the information is to be integrated (Jiang et al., 2001
, 2006
; Stein and Meredith, 1993
; Wallace and Stein, 1997
;). A number of critical factors must be met during postnatal life, and these will be discussed below. But first it is necessary to describe multisensory integration in its mature form, and outline the underlying circuits and computations used in this process.
Observations from Adult Animals
The visual, somatosensory, and auditory inputs to the deep SC of the cat are each represented in map-like fashion, and each map is in spatial register with the others (Stein et al., 1975
, 1976
, 1993
; Stein and Gallagher, 1981
; Middlebrooks and Knudsen, 1984
; Meredith and Stein, 1990
; Meredith et al., 1991
). The maps are formed by a clustering of the neurons responsive to stimuli from the same region of space defined by each neuron’s receptive field. Thus in the SC, neurons responsive to visual and/or auditory stimuli that have their receptive fields in frontal space are located in the rostral aspect of the structure. They are located near somatosensory neurons whose receptive fields are on the face. Visually responsive neurons with receptive fields in eccentric space are located toward the caudal aspect of the structure and abut auditory-responsive neurons whose receptive fields are also in eccentric spatial locations as well as somatosensory-responsive neurons whose receptive fields are toward the rear of the body. These overlapping sensory maps are also in spatial register with a premotor map that connects to the brainstem and spinal cord and represents a motor plan with which to respond (orient) to a stimulus. Therefore, regardless of which sense is providing information, an object or event in a given region of space activates neurons in the SC location that will drive orientation responses toward it – a highly efficient way to match incoming information from various sensory receptor organs with outgoing “motor” signals to the effectors controlling the orientation of those organs (Wurtz and Goldberg, 1971
; Harris, 1980
; Wurtz and Albano, 1980
; Stein and Clamann, 1981
; Grantyn and Grantyn, 1982
; Jay and Sparks, 1984
, 1987a
,b
; Sparks, 1986
; Peck, 1987
; Sparks and Nelson, 1987
; Guitton and Munoz, 1991
; Munoz and Wurtz, 1993a
,b
; Groh and Sparks, 1996a
,b
).
The spatial register among SC sensory maps is formed, in large part, by multisensory neurons that have multiple receptive fields, one for each sensory modality to which they respond. These receptive fields are in spatial register with one another (Meredith et al., 1991
, 1992
; Meredith and Stein, 1990
; King et al., 1996
; see also Gaither and Stein, 1979
; Newman and Hartline, 1981
; Knudsen and Brainard, 1991
; Gutfreund and Knudsen, 2004
; Zahar et al., 2009
), such that a visual–auditory neuron has its visual and auditory receptive fields overlapping one another in space (see Figure 1
). This cross-modal spatial register is one of the basic determinants of how multisensory information is synthesized (Stein and Meredith, 1993
), and the resultant modulation in SC signals is of primary interest here. Integrating signals from different senses can result in substantial increases or decreases in response magnitude (number of impulses). However, multisensory integration depends on a variety of factors, one of the most important of which is the relative location of the stimuli. When stimuli from different senses originate from roughly the same location at the same time (e.g., from the same event), they will fall within the overlapping receptive fields of a given multisensory neuron and can enhance its responses above those to either stimulus alone, and often above the sum of the two responses (again see Figure 1
).
Figure 1. A representative visual–auditory multisensory neuron. Top: visual space and visual receptive fields are shown on a hemisphere in which each concentric circle represents 10° of visual angle. The intersection of the horizontal and vertical meridians is the point directly in front of the animal. For auditory space, the full hemisphere represents frontal auditory space while the half-hemisphere represents caudal right space (for purposes of two-dimensional rendering, it has been split and folded forward). The visual stimulus was a small bar of light moved through the receptive field as indicated by the arrow, and the auditory stimulus was a broadband noise burst delivered from a speaker placed within the auditory receptive field (speaker location is depicted by the icon). Cross-modal stimuli consisted of the visual and auditory stimuli presented together. Visual and auditory receptive fields are shaded in blue and green. Bottom: Neuronal responses for visual (blue), auditory (green), and cross-modal (yellow) conditions are demonstrated with rasters and peristimulus time histograms. To the right: Bar graphs summarizing the mean response to all three conditions with the proportional enhancement observed in the cross-modal condition represented in orange.
These neural properties are seen not only in the SC of the cat (Meredith and Stein, 1983
, 1986a
,b
, 1996
; Stein et al., 1993
; Wallace et al., 1993
; Wallace and Stein, 1994
), but in the hamster (Meredith and Stein, 1983
, see also Stein and Dixon, 1979
), guinea pig (King and Palmer, 1985
), monkey (Wallace et al., 1996
), as well as in the cortex of cat (Wallace et al., 1992
), rat (Barth and Brett-Green, 2004
; Wallace et al., 2004a
), and monkey (Schroeder et al., 2001
; Schroeder and Foxe, 2002
, 2004
; Ghazanfar and Schroeder, 2006
; Lakatos et al., 2007
). One can also infer through functional imaging studies that similar neural properties exist in humans (Stein et al., 1996
; Hairston, 2001
; Laurienti et al., 2002
; Lovelace et al., 2003
; Calvert and Lewis, 2004
; de Gelder et al., 2004
; Fort and Giard, 2004
; Macaluso and Driver, 2004
; Sathian and Prather, 2004
).
The Importance of Cortical Influences in the Multisensory SC Circuit
It may seem surprising that the ability of a multisensory cat SC neuron to integrate its multiple sensory inputs, even when they are derived from the same place at the same time, is not a “given.” It depends on the presence of influences from cortex (see Figure 2
). This is not because those cortical influences render the SC neuron multisensory, as this is very rarely the case because SC neurons receive different sensory inputs from multiple sensory sources. Rather, it is because they are specifically required by the SC neuron to engage in multisensory integration. This conclusion was based on the results of a number of studies in cat showing that functional deactivation of the inputs from areas of association cortex disrupted multisensory integration in SC neurons but not their ability to respond to cues from different senses (Jiang et al., 2001
; Jiang and Stein, 2003
; Stein et al., 2002
; see also Burnett et al., 2007
). The relevant cortical areas in the cat are the anterior ectosylvian sulcus (AES) and the adjacent area, the rostral aspect of the lateral suprasylvian sulcus (rLS). Their homologues in the primate brain have not yet been identified.
Figure 2. The multisensory enhancement of many SC neurons depends exclusively on influences from AES. Top: the visual and auditory receptive fields of this neuron are depicted by the shaded areas on the illustrations of visual and auditory space with the same stimulus conventions as in Figure 1
. Bottom: (A) Shown are the neuron’s responses to the individual modality-specific stimuli and to their co-presentation before cortical deactivation (“control”). The movement of the visual stimulus is represented by the ramp labeled “V” and the auditory stimulus by the square wave labeled “A”. Below each stimulus trace are rasters and peristimulus time histograms that illustrate the neuron’s responses. Although the neuron showed a slightly better response to the auditory than to the visual stimulus, its best response was obtained when the two stimuli were presented in combination (“VA”). As shown in the summary bar graph at the far right, the multisensory response exceeded the dominant unisensory response by 147% and also exceeded the sum of its unisensory component responses (dashed line). (B) Multisensory response enhancement was eliminated by deactivating AES, but unisensory responses were not significantly affected. (C) Multisensory response enhancement was re-established 10 min after terminating cortical deactivation and initiating rewarming. (D,E) Deactivating and reactivating the rLS had no significant effect on the responses of this neuron. Significantly enhanced multisensory responses as compared to the dominant unisensory (auditory) response, t-test: *p < 0.05; **p < 0.01. Adapted from Jiang et al. (2001
)
These association cortical areas are important in this context, as other cortical inputs to the cat SC do not appear to be specifically important for multisensory integration (Wilkinson et al., 1996
; Jiang et al., 2007
). Of the two, the most important are the inputs from AES. These inputs are unisensory and match the modality-convergence profile of the SC neuron they target (Wallace et al., 1992
). In other words, a visual–auditory SC neuron that can integrate its inputs will receive converging inputs from the visual and auditory subdivisions of AES that will match visual and auditory inputs the SC neuron receives from non-AES sources (Figure 2
).
However, at birth, neither the cat nor the monkey SC contains neurons capable of integrating cross-modal stimuli (Wallace and Stein, 1997
, 2001
). Because the SC develops earlier than does the cerebral cortex (see Stein et al., 1973a,
b
), it is a good bet that higher order areas of the brain are also incapable of synthesizing sensory information from different modalities, a supposition borne out by recent studies in cortex (Wallace et al., 2006
; Carriere et al., 2007
). Recent perceptual studies in humans also support the idea that the integrative capacity of the brain appears to develop over time (Neil et al., 2006
; Putzar et al., 2007
).
Because the cat is born at a comparatively early stage of maturation, it was a good model within which to examine this process. At birth, its SC neurons appear highly immature and all of them, even those in its multisensory layers, are unisensory (Stein et al., 1973; Wallace and Stein, 1997
). The only sensory-responsive SC neurons at this stage of life are somatosensory (Stein et al., 1973), and these neurons first appear prenatally, presumably in part, to prepare the neonate with the perioral responsiveness necessary to facilitate its quest for the nipple (Larson and Stein, 1984
). The ear canals and the eyes are closed at birth, and auditory neurons do not begin appearing until about five postnatal days (see Stein et al., 1973), and visual neurons do not begin appearing until about three postnatal weeks (Kao et al., 1994
). Neonatal SC neurons have very large receptive fields; they rapidly habituate to stimuli, and do not yet show the high selectivity for the stimulus parameters (e.g., velocity, direction of movement) that are characteristic in the adult. Their receptive fields shrink and their stimulus selectivity matures over the next few weeks of life. However, their multisensory development is far more protracted.
An inherent distinction between multisensory and unisensory processes should be noted here, as it is relevant to what one can expect of their development. Multisensory processes are unique in that they are integrating independent channels of information (Alvarado et al., 2007a
,b
; Ernst and Banks, 2002
; Gingras et al., 2006
; Rowland et al., 2007
). Thus, while what we know about unisensory development helps define the likely period of sensory maturation, we cannot extrapolate from that the principles governing multisensory maturation. A case in point in the maturation of visual cortex neurons is their development of specialization for a “preferred” orientation. When exposed to all line orientations or complex forms, different subpopulations of visual cortical neurons ultimately “prefer” a given line orientation, with all orientations represented among the specialized neuronal subgroups. In contrast, the end-point in the maturation of multisensory integration in both SC and cortical neurons is a more generalized product. Multisensory neurons are likely to be designed to extract the statistical regularities in space and time between the component stimuli in cross-modal events so that they can establish general principles to guide responses to these events.
The first multisensory neuronal type to develop in the SC is the somatosensory–auditory neuron. These neurons do not begin appearing until approximately postnatal day 10, very soon after the deep SC is capable of responding to auditory cues. Visual–nonvisual neurons begin appearing at three postnatal weeks, as soon as the deep SC begins responding to visual cues (the purely visual overlying layers develop considerably earlier, see Stein et al., 1973; Kao et al., 1994
). All of these new multisensory neurons lack the ability to engage in multisensory integration. They can respond to the different sensory inputs individually, but when presented with spatiotemporally concordant cross-modal stimuli, cannot integrate across them to generate enhanced responses. This capacity requires a far longer developmental time course (2 to 3+ months), during which there is considerable maturation in cortico-collicular projections (Stein and Gallagher, 1981
; Wallace and Stein, 1997
, 2000
; Stein et al., 2002
).
Why is it so Protracted?
The long postnatal time course of multisensory maturation suggests that this capacity does not depend on a passive process that simply unfolds over time. This is because the developmental period is one in which the brain is also acquiring a great deal of sensory experience. Could it be that the prolonged maturation of this capacity is specifically because such experiences encourage it and direct it? We believe so. By extracting the statistical regularities of early life experiences with cross-modal stimuli, the brain can adapt multisensory integration to the demands of the specific environment in which it will be used.
This, of course, requires that multisensory integration is plastic, something generally associated more with cortical than midbrain processes, and suggests that cross-modal experiences may be instantiated in the SC via the cortico-collicular pathway. Given the well-known sensitivity of cortex to sensory experience (Buonomano and Merzenich, 1998
) and the comparative insensitivity of the SC to such experiences (Wickelgren and Sterling, 1969
), as well as the fact that cortical influences are critical for SC multisensory integration, this seemed to be a highly likely possibility. Therefore, a multi-pronged approach was designed to explore these and other related issues by examining: (1) the effects of rearing animals without the possibility of experience with certain cross-modal cues (e.g., visual–nonvisual); (2) the effects of rearing animals by perturbing the spatial concordance of the cross-modal cues normally associated with the same event; and (3) the maturation of SC multisensory integration either in the absence of its critical cortical inputs or by deactivating (reversibly) this portion of the multisensory SC circuit so that that it would not have access to the sensory experience normally acquired during early life.
Dark Rearing
eliminates the possibility of obtaining early visual–nonvisual experience. It therefore affords the opportunity to examine how the absence of such experience affects the development of visual–nonvisual integration and SC function (Wallace et al., 2004b
). After rearing cats from birth to 6 months in that way, the SC was found to have developed all of its characteristic unisensory neurons (visual, auditory, somatosensory) and populations of each possible multisensory neuronal subtype. Furthermore, the relative size of each of these populations was not very different from that found in the normal SC. But, these neurons had very large receptive fields that were more like the neonatal state than the adult state (Figure 3
). The neurons were also incapable of multisensory response enhancement. So, with respect to both their size and lack of multisensory integration capability, they seemed neonatal, as if their development had been arrested. This was consistent with several possibilities (e.g., light exposure might be essential for multisensory integration), the most compelling of which appeared to be that experience with a given combination of senses is essential for the maturation of multisensory integration capabilities. As noted earlier, this might be because the system evolved in a way that requires experience with cross-modal stimuli to adapt multisensory integration to fit the presumptive sensory environment in which it will be used. In the absence of this experience, the development does not proceed. If true, this would also mean that very different cross-modal experiences will produce very different multisensory products.
Figure 3. (A) Multisensory integration, which is present in SC neurons in animals raised in a normal illuminated environment, is significantly diminished in animals raised in complete darkness. Rasters and peristimulus time histograms show the responses of two neurons, one from a normal control and one from a dark-reared animal, to both modality-specific and cross-modal stimuli. Summary bar graphs illustrate the mean responses for each of the conditions and the magnitude of the multisensory interaction. The “sum” represents the response predicted based on the addition of the two unisensory component responses. *p < 0.05. “A,” Auditory; “V,” visual; “VA,” visual–auditory. (B) Receptive fields in dark-reared animals were significantly larger than those in control animals. Receptive fields (shading) are shown for three representative multisensory neurons in control (left) and dark-reared (right) animals. Conventions are the same as in Figure 2
. The central bar graph plots the relative sizes for the visual, auditory, and somatosensory receptive fields of multisensory neurons in the SC of normal (gray bars) and dark-reared (black bars) animals. Note that these mean measures are standardized to receptive field sizes in normal adults (100%). Values in parentheses represent the number of neurons in each group. *p < 0.05; **p < 0.01. Adapted from Wallace et al. (2004)
Spatial-Disparity Rearing is one way to test whether the basic properties of multisensory integration will adapt to the specifics of early experience. Animals were raised from birth to 6 months of age in a dark room in which visual–auditory stimuli were presented periodically. The stimuli were synchronous, as if derived from the same event, but they were spatially disparate. The speakers and lights from which they were elicited were fixed to different spots on the wall. Most SC neurons of these animals appeared to be insensitive to this rearing condition. They had large receptive fields and were unable to engage in multisensory integration (Figure 4
), and looked just like those of dark-reared or neonatal animals (Wallace and Stein, 2007
). However, many showed the impact of this experience by developing poorly aligned visual–auditory receptive fields, some of which were completely out of spatial register with one another as they had no areas in common (see Figure 4
). This has not been seen in the SC of cats raised in a normal environment or those raised in the dark. As a result of this “anomalous” development, only those visual–auditory stimuli that were highly disparate in space could fall simultaneously in the respective receptive fields of these multisensory neurons, and when they did, they enhanced the magnitude of the neuron’s response. This represented a reversal of the normal “spatial principle” of multisensory integration in which only spatially congruent cross-modal stimuli yield response enhancement, and spatially disparate stimuli either yield response depression or have no effect on one another (see Meredith and Stein, 1996
; Kadunce et al., 1997
).
Figure 4. Animals reared with spatially disparate visual and auditory stimuli develop atypical receptive fields and multisensory integration. (A) Receptive field overlap in neurons from normal and spatial disparity reared animals. Top: A sampling of all SC quadrants (left) yielded visual (red) and auditory (green) receptive fields whose center distributions are shown (right). Bottom: % receptive field overlap in control animals (yellow) was often 91–100%; far exceeding that seen in animals reared with spatially disparate visual and auditory stimuli (red, often <10%). (B) An example of the non-overlapping receptive fields of an animal reared with visual–auditory spatial disparity. The visual receptive field is in blue and the auditory in green. Note the atypical multisensory integration: when visual and auditory stimuli were placed within their respective receptive fields they were spatially disparate, but produced significant (<0.05, t-test) enhancement. This is a striking reversal of the normal condition wherein these receptive fields overlap one another and spatially coincident stimuli are required for multisensory integration. However, this result is consistent with the animal’s abnormal multisensory experience. Adapted from Wallace and Stein (2007)
.
This result strongly supported the contention that experience with cross-modal cues early in life crafts the neural circuits underlying multisensory integration. That association cortex is essential for guiding this process is suggested by two observations: first, its presence during development is necessary for SC multisensory integration to appear; second, the maturation of its influences on SC neurons parallels their development of multisensory integration capabilities (Wallace and Stein, 2000
).
Developing an SC without input from association cortex is a tactic used by Jiang et al. (2006)
to assess the normal impact of these descending influences on guiding the maturation of SC multisensory integration. As it turns out, neonatal removal of this cortex precluded the maturation of SC multisensory integration. Its influence appeared to be critical, as other cortical areas did not compensate for its loss. Interestingly, however, both AES and rLS had to be removed for this to happen. If only one of the areas was removed, compensatory plasticity was induced in the other, and most SC neurons developed their characteristic multisensory capabilities (Jiang et al., 2006
). These experiments provided another important observation. In the absence of these cortical inputs, multisensory SC neurons develop poor alignment among their different receptive fields (Figure 5
). Apparently, association cortex plays an important role in guiding SC receptive field alignment and, thereby, constraining how stimuli will be processed during multisensory integration, even when those receptive fields involve inputs derived from other afferent sources.
Figure 5. Three characteristic examples of the loss of visual–auditory receptive field overlap following combined lesions of the anterior ectosylvian sulcus (AES) and the rostral lateral suprasylvian (rLS). The bar graphs represent the percentage of visual–auditory receptive field register in SC neurons. There is a high degree of receptive field overlap in normal controls, however that overlap was significantly degraded in animals with combined AES and rLS lesions, but not in those with lesions restricted to AES or rLS; vertical bars SE, *p < 0.05). Adapted from Jiang et al. (2006)
.
Presumably, the influences of association cortex ensure that SC multisensory processes properly reflect the physical configurations of the cross-modal events that have been experienced regardless of whether these are typical (e.g., “normal”) or atypical. In the absence of cortex there is no way to translate experience with cross-modal cues so that the default condition approximates the neonatal state in which there is no multisensory integration.
Chronic Cortical Deactivation during Development
Perhaps the most powerful evidence for this integral role of association cortex in the maturation of SC multisensory integration comes from experiments that are still ongoing (Stein and Rowland, 2007
; Stein et al., 2008
). These experiments involve rendering the neonatal association cortex unable to gather early cross-modal experiences (see Rowland et al., 2005
; Stein and Rowland, 2007
; Stein et al., 2008
). This is accomplished by implantation of muscimol-impregnated elvax pledgets that block intrinsic cortical activity for many weeks. Once the muscimol stores are depleted, or the pledgets are removed, cortex becomes active again. Preliminary observations indicate that such a disruption of cortical activity during the period in which multisensory integration normally develops interferes with the process. SC neurons still become multisensory (i.e., they respond to more than one sensory modality), and develop their characteristic unisensory properties, but are unable to integrate cross-modal information to enhance SC-mediated behaviors, even after cortex is reactivated. These observations strongly point to cortex as the portal through which cross-modal experience has access to the brain’s multisensory circuitry.
Development/Maintenance of Multisensory Integration in the Adult
In the chronic cortical deactivation experiments, the cortex was deactivated unilaterally and the deficits were, as expected, confined to ipsilateral SC neurons and to multisensory responses to stimuli in contralateral sensory space. However, in retests after 4 years of normal experience, there was a startling result. The previous deficits disappeared, suggesting that appropriate sensory experience in adulthood could compensate for cross-modal anomalies developed early in life. This is of more than passing interest in the current context or to those with early visual and/or auditory deficits that can be corrected later in life via drug therapy or prosthetic devices. The possibility of adult plasticity in this context also raises interesting questions about the requirements for maintaining normal multisensory integration in adulthood. We have already found that normal adult cats placed in the dark for up to 6 months retain their normal capability for multisensory integration (unpublished observations), indicating that it can be retained without constant updating and in the absence of conflicting information. It would, however, be of considerable interest to determine if altering the rules of, for example, visual–auditory events, as in disparity rearing, would produce a change in the principles of integration. This would suggest an enduring plasticity in this function, albeit one that might change with age and experience.
Some Implications
Nevertheless, for those individuals with anomalies of sensory integration and/or sensorimotor transduction, the plastic nature of these processes throughout life offers promise for successful therapeutic intervention. Because of the general role of the SC in integrating information from multiple senses to detect external events and initiate adaptive responses to them, it is highly likely that it plays a role in the manifestations of whatever lies at the neurological core of SPD. However, any anomalies in these overt behaviors will impact the feedback generated from caregivers and others, as well as the individual’s internal representations of external events and peripersonal space. These could contribute to the established difficulties in dealing appropriately with external stimuli, especially when multiple sensory cues are being generated in complex environments that require a rapid choice among competing alternatives. The potentially destructive cycle is obvious. At present, many of the ameliorative techniques that deal with the symptoms of SPD, which are discussed in other contributions to this special issue of Frontiers in Integrative Neuroscience, are likely dealing in large part with the circuit underlying their behavioral expression. This would include the cortical–collicular axis. In this vein it would be interesting to consider how much of the practical success that has already been achieved in dealing with this condition, via physical and occupational therapy, reflects alterations in this circuit. Unfortunately, our current understanding of the neurological basis of SPD is rudimentary compared to that of many other developmental disorders. Only by addressing the many issues raised in this special issue of the Journal will we begin to understand its origin and expression and be able develop the most effective strategies for minimizing its disruptive consequences.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The research described was supported in part by NIH grants EY016716 and NS036916, and a grant from the Wallace Research Foundation.
Miller, L. J., Reisman, J. E., McIntosh, D. N., and Simon, J. (2001). An ecological model of sensory modulation: performance of children with fragile X syndrome, autism, attention-deficit/hyperactivity disorder, and sensory modulation disorders. In Understanding the Nature of Sensory Integration With Diverse Populations, S. S. Roley, E. I. Blanche and R. C. Schaaf, eds (San Antonio, TX, Therapy Skill Builders), pp. 57–88.