# DEVELOPING SYNAESTHESIA

EDITED BY: Nicolas Rothen, Julia Simner and Beat Meier PUBLISHED IN: Frontiers in Human Neuroscience

#### *Frontiers Copyright Statement*

*© Copyright 2007-2015 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

*All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88919-579-4 DOI 10.3389/978-2-88919-579-4

## About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

## Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

## Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

## What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

## **DEVELOPING SYNAESTHESIA**

## Topic Editors:

**Nicolas Rothen,** University of Sussex, UK **Julia Simner,** University of Edinburgh, UK **Beat Meier,** University of Bern, Switzerland

Developing synaesthesia: a subset of the figures from the different contributions in this Research Topic

Copyright: Mroczko-Wa˛sowicz and Nikolić, 2014; Terhune at al., 2014; Sinke et al., 2014; Asano and Yokosawa, 2013

Synaesthesia is a condition in which a stimulus elicits an additional subjective experience. For example, the letter E printed in black (the inducer) may trigger an additional colour experience as a concurrent (e.g., blue). Synaesthesia tends to run in families and thus, a genetic component is likely. However, given that the stimuli that typically induce synaesthesia are cultural artefacts, a learning component must also be involved. Moreover, there is evidence that synaesthetic experiences not only activate brain areas typically involved in processing sensory input of the concurrent modality; synaesthesia seems to cause a structural reorganisation of the brain. Attempts to train non-synaesthetes with synaesthetic associations have been successful in mimicking certain behavioural aspects and posthypnotic induction of synaesthetic experiences in non-synaesthetes has even led to the according phenomenological reports. These latter findings suggest that structural brain reorganization may not be a critical precondition, but rather a consequence of the sustained coupling

of inducers and concurrents. Interestingly, synaesthetes seem to be able to easily transfer synaesthetic experiences to novel stimuli.

Beyond this, certain drugs (e.g., LSD) can lead to synaesthesia-like experiences and may provide additional insights into the neurobiological basis of the condition. Furthermore, brain damage can both lead to a sudden presence of synaesthetic experiences in previously non-synaesthetic individuals and a sudden absence of synaesthesia in previously synaesthetic individuals. Moreover, enduring sensory substitution has been effective in inducing a kind of acquired synaesthesia.

Besides informing us about the cognitive mechanisms of synaesthesia, synaesthesia research is relevant for more general questions, for example about consciousness such as the binding problem, about crossmodal correspondences and about how individual differences in perceiving and experiencing the world develop. Hence the aim of the current Research Topic is to provide novel insights into the development of synaesthesia both in its genuine and acquired form. We welcome novel experimental work and theoretical contributions (e.g., review and opinion articles) focussing on factors such as brain maturation, learning, training, hypnosis, drugs, sensory substitution and brain damage and their relation to the development of any form of synaesthesia.

Nicolas Rothen and Beat Meier initiated and managed the research topic. Nicolas Rothen, Beat Meier, and Julia Simner edited articles of the research topic.

**Citation:** Rothen, N., Simner, J., Meier, B., eds. (2015). Developing Synaesthesia. Lausanne: Frontiers Media. doi: 10.3389/978-2-88919-579-4

## Table of Contents


*144 Can grapheme-color synesthesia be induced by hypnosis?*

Hazel P. Anderson, Anil K. Seth, Zoltan Dienes and Jamie Ward

*153 A longitudinal study of grapheme-color synesthesia in childhood: 6/7 years to 10/11 years*

Julia Simner and Angela E. Bain

*162 Developmental aspects of synaesthesia across the adult lifespan* Beat Meier, Nicolas Rothen and Stefan Walter

## Developing synaesthesia: a primer

#### Beat Meier <sup>1</sup> \* and Nicolas Rothen<sup>2</sup>

1 Institute of Psychology and Center for Learning, Memory, and Cognition, University of Bern, Bern, Switzerland, <sup>2</sup> Institute of Psychology, University of Sussex, Brighton, UK

Keywords: synaesthesia/synesthesia, immune system, serotonin, GABA antagonists, development, neonatal, multisensory integration, connectivity

Synaesthesia is a variation of human experience that involves the automatic activation of unusual concurrent experiences in response to ordinary inducing stimuli. The causes for the development of synaesthesia are not well understood yet. Synaesthesia may have a genetic basis resulting in enhanced cortical connectivity during development. However, in some cases synaesthesia has a sudden onset, for example, caused by posthypnotic suggestions, drug exposure, or brain injury. Moreover, associative learning during a critical developmental period also seems to play an important role. Synaesthesia may even be acquired by training in adulthood. In this research topic, we bring together topical hypotheses, theories and empirical studies about the development of synaesthesia.

According to the immune hypothesis introduced by Carmichael and Simner (2013), the genes that are responsible for normal cortical development are also involved in the development of synaesthesia. As many of these genes have a function for both the immune system and for altering connectivity via axonal guidance, synaptic connectivity, and pruning, the interaction between the central nervous system and the immune system during early life may play a pivotal role in the development of synaesthesia. A common genetic basis of synaesthesia and autism spectrum disorders may also explain the higher prevalence of synaesthetes among patients diagnosed with autism spectrum disorder reported by Neufeld et al. (2013).

Likewise based on a neurochemical basis, Brogaard (2013) put forward the serotonergic hyperactivity hypothesis for the development of synaesthesia. Accordingly, excessive serotonin in the brain may be a common cause for synaesthesia induced by psychedelic drugs, brain injury, and in individuals with autism. Motivated by similar assumptions, Terhune et al. (2014) used a psychophysical approach to test this hypothesis. They found no support, neither for the serotinergic hyperactivity hypothesis nor for a related, reduced GABA levels hypothesis. Similarly, null findings were reported by Sinke et al. (2014) in an investigation on the relationship between multisensory integration and event related potentials. Nevertheless, consistent with previous research, they found evidence for alterations in early visual processing in synaesthesia.

According to the neonatal hypothesis synaesthetic associations between basic shapes and colors may be present already early in childhood. Brang et al. (2013) hypothesized that even when these associations can be refined by experience they can interfere with learning novel shape-color associations later. In an empirical study they found support for this hypothesis. Other determinants of synaesthetic associations include ordinality (e.g., the position in a grapheme sequence; e.g., "c" is the third letter in the alphabet) and sound. In a Japanese sample of synaesthetes, Asano and Yokosawa (2013) addressed the relative impact of ordinality, sound, and shape as potential factors that might determine the specific color of a grapheme in English and in Japanese (Hiragana). For Hiragana characters, they found that ordinality, sound and shape all contributed to the prediction of color attributes while for English only ordinality and shape were predictive. These results underline the important role of ordinality due to its primacy when learning the graphemes in childhood. Notably, ordinality can be considered as conceptual information.

Despite the notion that synaesthesia is a perceptual phenomenon, Mroczko-W ˛asowicz and Nikolic (2014) ´ emphasize that the stimuli that trigger synaesthesia (i.e., the inducers) often take

#### Edited and reviewed by:

Hauke R. Heekeren, Freie Universität Berlin, Germany

> \*Correspondence: Beat Meier, beat.meier@psy.unibe.ch

Received: 12 February 2015 Accepted: 31 March 2015 Published: 20 April 2015

#### Citation:

Meier B and Rothen N (2015) Developing synaesthesia: a primer. Front. Hum. Neurosci. 9:211. doi: 10.3389/fnhum.2015.00211 the form of concepts and only the synaesthetic experiences (i.e., the concurrents) typically exhibit clear perceptual properties. This also holds for sequence-space synaesthesia, in which ordinal elements are experienced as occupying locations in extended areas of space. According to Price and Pearson (2013) this may very well be a variety of normal visuospatial imagery that has evolved as a mnemonic strategy to overcome developmental retardation of the phonological loop. Interestingly, an extended phenomenological investigation of spatial-form synaesthesia in a single case showed that often the synaesthetic experience followed an internal verbalisation of the inducer, indicating an auditory component to this form of synaesthesia (Gould et al., 2014). Moreover, the insights into and the kinds of synaesthetic experiences seemed to expand during the interview (cf. Price, 2014 for further considerations). These findings are in line with the learning theory of synaesthesia put forward by Watson et al. (2014). They emphasize the mutual influences of learning and synaesthesia and suggest that it may have evolved (and survived) because it is useful for learning.

A related question concerns whether synaesthesia can be induced by training and hypnosis. In a comprehensive review, Rothen and Meier (2014) conclude that behavioral facets of synaesthesia can be induced by training, but so far there is little evidence that this is also accompanied by a "synaesthetic" experience. In their commentary, Colizoli et al. (2014) ask for additional training studies which should be evaluated according to a diagnostic checklist. Specifically, they suggest that for each trained individual, consistency, bandwidth, automaticity, conscious experience, perceptual nature, and presence should

## References


be assessed, and potential demand effects controlled. Anderson et al. (2014) used hypnosis to investigate a potential performance advantage in a visual search task previously reported for genuine grapheme-color-synaesthetes. This did not materialize. However, accurate responses were associated with reports of more intense colors, suggesting that colors induced by posthypnotical suggestions were only perceived after successful target detection.

Last, but not least, two studies directly addressed the development of synaesthesia, that is, the consistency of inducerconcurrent pairs in grapheme-color synaesthesia across childhood development and the adult lifespan. In a longitudinal study, Simner and Bain (2013) found a protracted developmental trajectory from age 6 to age 11 with an age-related increase of the number of consistent grapheme-color associations. Meier et al. (2014) used a cross-sectional approach to investigate age-related changes in more than 400 grapheme-color synaesthetes aged 18– 91. They found a decrease of the number of consistent graphemecolor association in older age. Together these findings suggest that synaesthesia follows a similar developmental trajectory as many other cognitive functions.

Overall, this Research Topic provides a comprehensive overview of different approaches available to address the development of synaesthesia. There may be large individual differences in the developmental trajectory and across different forms of synaesthesia. So far, the focus was mainly on the development of grapheme-color- and sequence-space synaesthesia which both involve cultural artifacts as inducers. Future studies should also address the development of other forms, for example, soundcolor synaesthesia.

form synaesthesia." Front. Hum. Neurosci. 8:439. doi: 10.3389/fnhum.2014. 00439


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2015 Meier and Rothen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## The immune hypothesis of synesthesia

## *Duncan A. Carmichael 1,2,3\* and Julia Simner <sup>1</sup>*

*<sup>1</sup> Department of Psychology, University of Edinburgh, Edinburgh, UK*

*<sup>2</sup> Institute for Adaptive and Neural Computation, University of Edinburgh, Edinburgh, UK*

*<sup>3</sup> Division of Psychiatry, Royal Edinburgh Hospital, University of Edinburgh, Edinburgh, UK*

*\*Correspondence: d.a.carmichael@ed.ac.uk*

#### *Edited by:*

*Nicolas Rothen, University of Sussex, UK*

#### *Reviewed by:*

*Berit Brogaard, University of Missouri, USA*

**Keywords: synesthesia, genetics, immunity, neurological basis, genes, cortical connectivity, immune system**

Synesthesia is a hereditary, neurological condition in which a wide range of common stimuli (e.g., letters, sounds, flavors) trigger unexpected secondary sensations, for example, synesthetes listening to music might see colors in addition to hearing sounds (Ward et al., 2006; see Simner and Hubbard, 2013, for review). Current explanations of synesthesia posit structural and/or functional differences in the synesthete brains, and frame their models in terms of excess cortical connectivity or altered cortical feedback. Here, we propose an immune hypothesis of synesthesia, which supplements existing models by suggesting how such altered connectivity may arise and how associations between synesthesia and other conditions might be explained.

Two categories of model seek to explain the generation of synesthetic experiences, and more recent models are a hybrid of both (Brang et al., 2010). The *crossactivation model* (Ramachandran and Hubbard, 2001) suggests that excess connectivity between functional areas of the cortex allows activation in one cortical area (e.g., auditory cortex) to directly trigger activation in another (e.g., visual cortex). Evidence in support of this model comes for example from diffusion tensor imaging (DTI) and shows that excessive connectivity is indeed a feature of the synesthetic brain (Rouw and Scholte, 2007). *Re-entrant* and *disinhibited feedback models* propose that synesthetic sensations are caused by disinhibited feedback from higher cortical areas (e.g., in parietal lobe) failing to suppress non-relevant activation from lower cortical areas (Grossenbacher and Lovelace, 2001). This type of disinhibited feedback may result from excessive activity of excitatory neurons within the delicate balance between both excitatory and inhibitory neurons in the brain (Hubbard et al., 2011). Despite appearing superficially different, connectivity and feedback models need not be mutually exclusive. It is unlikely that altered feedback happens entirely in the absence of changes in cortical connectivity, given the Hebbian principle that simultaneous activity strengthens interconnectivity between neurons. Therefore, these two approaches might be considered somewhat unified in that connectivity models propose aberrant connectivity as the primary causal mechanism underlying synesthesia whereas feedback models might allow altered connectivity as an *indirect* consequence of disinhibited feedback.

While these models are now more than a decade old, explanations of *how* these cortical characteristics might arise have proven elusive thus far (but see Brang and Ramachandran, 2008; Mitchell, 2013) and we explore this here. Synesthesia is thought to be primarily neurodevelopmental in nature (Spector and Maurer, 2009). Consequentially, known processes of brain development are likely to be implicated in its emergence. We propose that insight might be gained from examining the functionality of genes that regulate the types of altered synesthetic cortical connectivity assumed in these models above (i.e., genes for axon guidance, synapse density). This is the approach we follow here.

#### **DUAL GENE FUNCTIONALITY: CONNECTIVITY AND IMMUNITY**

Above, we saw that current models link synesthesia to altered structural connectivity, misregulated feedback mechanisms, or a combination thereof. Explaining how synesthesia develops might therefore come from considering the developmental processes responsible for cortical connectivity. The immune system is known to play a significant role in these processes (for review, see Boulanger, 2009) and we ask here whether a propensity to develop synesthesia may be linked to the expression of immune proteins in the CNS, since this expression can be conferred by genes with functions of both immunity and cortical development. Indeed, many genes have been shown to have precisely this dual functionality (Boulanger, 2009). As well as their immunity function, such genes act in cortical development, altering structural and/or functional connectivity by influencing the development of axonal guidance, synaptic connectivity, and synaptic pruning. One outcome of these changes may therefore be the anomalous pattern of connectivity proposed by the crossactivation theory in the development of synesthesia. Alternatively, the immune system could have a direct influence on excitatory neuronal activity, leading to the outcomes proposed by disinhibited feedback models. This is because the immune system plays an important role in the initial development and subsequent plasticity of glutamatergic synapses, the primary excitatory transmission pathway in the mammalian cortex (Fourgeaud and Boulanger, 2010).

#### **CAN THE IMMUNE SYSTEM INFLUENCE THE DEVELOPMENT OF THE BRAIN?**

Is it plausible to make a link between the immune system and regulation of the central nervous system (CNS)? Isolated from the rest of the body by the blood-brain-barrier, the CNS was once thought to barely interact with the immune system, leading to the long held view that the CNS was "immune privileged" (McAllister and van de Water, 2009). However, research now shows a complex communication between the CNS and immune system, with wide-reaching consequences for brain regulation and development, both in health and disease (Elmer and McAllister, 2012). Immune proteins are known to play a role at many stages in the developmental pathway. They are integral components of phases critical to brain development and plasticity, such as neuronal guidance, synapse development, and synaptic remodeling (Boulanger, 2009). We therefore hypothesize that CNS and immune system interaction may be the biological mechanism which confers the predisposition to develop synesthesia.

Which aspects of the immune system are known to exhibit the type of dual functionality under discussion in this article (i.e., functionality in both immunity and cortical development)? Several areas of this extraordinarily complicated system are worth highlighting. The *complement system* is one possible candidate a complex cascade of protein interactions involved in immunity which has also been shown to play an important role in tagging of synapses to be eliminated by pruning during development (Stephan et al., 2012). Another candidate relates to cytokines, which are immune proteins that have also been shown to play significant roles in neurogenesis and synaptic plasticity (Bauer et al., 2007). A third candidate relates to major histocompatibility complex (MHC) proteins, which are an integral part of the adaptive immune system found on the surface of the majority of nucleated cells and widely expressed in neurons of the CNS (Boulanger, 2004). In addition to fulfilling a crucial function in immune response, MHC class I molecules and related components are thought to be involved in a range of developmental processes, such as activitydependent plasticity and synaptic refinement (Boulanger, 2009). The MHC locus contains several hundred genes, and has also been widely implicated in a range of autoimmune conditions, such as multiple sclerosis (MS), irritable bowel syndrome (IBS), and rheumatoid arthritis (Fernando et al., 2008). We point out, however that the immune system consists of many hundreds of individual factors and processes. Given this, our suggestions above should be considered speculative and by no means exhaustive. Nevertheless, we consider them to each be plausible candidates for future investigation.

We end this section by asking whether existing studies into the genetics of synesthesia would support our immunity hypothesis. In other words, have they identified areas of the genome containing immune system genes? Research into synesthesia genetics is in its infancy and as yet, there are insufficient data to draw firm conclusions. No synesthesia genes have yet been identified and no firm mode of inheritance has yet been elucidated. However, evidence from the two existing studies on the genetics of synesthesia (Asher et al., 2009; Tomson et al., 2011) have identified several chromosomal regions of interest, and these regions do contain immune function genes. Asher et al. (2009) found significant linkage to chromosome 2q24 and possible linkage to areas on other chromosomes (5q33, 6p12, and 12p12), while Tomson et al. (2011) identified a candidate region on chromosome 16q12.2-23.1. The authors of both studies draw the conclusion that synesthesia is likely to be a condition influenced by a variety of genes in multiple loci. Nonetheless, the chromosomal regions of interest highlighted in these two investigations do contain immune function genes (e.g., interleukin-17, a cytokine protein found on chromosome 6p12), although we wish to be clear that many other viable candidates also lie outwith these regions.

## **THE IMMUNE HYPOTHESIS AS A FRAMEWORK FOR THE STUDY OF CO-MORBIDITIES**

An immune hypothesis of synesthesia might additionally explain recent comorbidity data which suggests that having synesthesia may be associated with increased risk of other clinical conditions. Carruthers et al. (2012) report an association between synesthesia and IBS, having found an elevated prevalence of synesthesia in a population of people with IBS. Other researchers have raised the possibility that synesthesia may also be found at elevated rates within populations with autism (Baron-Cohen et al., 2007) or migraine (Alstadhaug and Benjaminsen, 2010). The immune system plays a prominent role in all of these conditions (Collins, 2002; Bruno et al., 2007; Enstrom et al., 2009), suggesting that altered immune system function may be a common causal link. If so, the immune model proposes a plausible framework by which to investigate co-morbidity between synesthesia and other conditions. If this hypothesis is correct, we might ask whether the prevalence of synaesthesia is also higher in populations with other autoimmune conditions. Indeed, recent data from our lab has led us to explore whether developmental synaesthesia might occur more prevalently in people with the radiological profile of multiple sclerosis (MS), for example, a demyelinating disease of the human CNS (Simner et al., submitted). A maladaptive immune system is an undisputed factor in the pathogenesis of MS (Trapp and Nave, 2008), and the majority of genes implicated in MS have an immune function (Gourraud et al., 2012). The immune hypothesis of synaesthesia might therefore lead us to investigate whether synaesthesia and autoimmune conditions such as MS could share overlapping genetic origins in contributing to cortical development and immune function.

## **CONCLUSION AND FUTURE DIRECTIONS**

We have proposed that CNS/immune system interactions during early life may play a role in the development of synesthesia. We have asked whether genes with dual functionality in brain development and immunity may be at the origin of existing models of synesthesia, and this mechanism would provide a framework to investigate associations between synesthesia and other immune-related conditions. We make our proposal here as a model for developmental synesthesia, although not all cases of synesthesia are developmental in nature. Synesthesia may also be acquired, for example as a result of brain injury (Schweizer et al., 2013) or induced by consumption of psychoactive drugs such as Lysergic acid diethylamide (LSD; e.g., Cytowic, 1989). Our hypothesis does not speak directly to such cases, and it is not yet known whether these different forms of synesthesia have the same neural origins or mechanisms. It is interesting to note however that immune system activity is elevated after brain injury, and processes such as apoptosis do become activated (Griffiths et al., 2010). It is therefore at least plausible to ask whether the immune system might also play a role in the appearance of non-developmental synesthesias. Identification of genes that contribute to the development of synesthesia will make a significant contribution to the validity of this hypothesis, and whether synesthesia has one cause or many.

## **ACKNOWLEDGMENTS**

We are grateful to Richard Shillcock for useful discussions, Lisa Boulanger for insightful suggestions and feedback and the anonymous reviewers for helpful comments on the manuscript. Duncan A. Carmichael was supported in part by grants EP/F500385/1 and BB/F529254/1 for the University of Edinburgh, School of Informatics Doctoral Training Centre in Neuroinformatics and Computational Neuroscience (www*.*anc*.*ac*.*uk/dtc) from the UK Engineering and Physical Sciences Research Council (EPSRC), UK Biotechnology and Biological Sciences Research Council (BBSRC), and the UK Medical Research Council (MRC).

## **REFERENCES**


*Received: 31 July 2013; accepted: 23 August 2013; published online: 11 September 2013.*

*Citation: Carmichael DA and Simner J (2013) The immune hypothesis of synesthesia. Front. Hum. Neurosci. 7:563. doi: 10.3389/fnhum.2013.00563*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2013 Carmichael and Simner. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Is synesthesia more common in patients with Asperger syndrome?

#### *Janina Neufeld1, Mandy Roy2, Antonia Zapf 3, Christopher Sinke4, Hinderk M. Emrich2, Vanessa Prox-Vagedes 2, Wolfgang Dillo2 and Markus Zedler <sup>2</sup> \**

*<sup>1</sup> Centre for Integrative Neuroscience and Neurodynamics, School of Psychology and Clinical Language Sciences, University of Reading, Reading, UK*

*<sup>3</sup> Deparment of Medical Statistics, Georg-August-University, Goettingen, Germany*

*<sup>4</sup> Department of Neurology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany*

#### *Edited by:*

*Nicolas Rothen, University of Sussex, UK*

#### *Reviewed by:*

*Simon Baron-Cohen, University of Cambridge, UK Laurent Mottron, hôpital Rivière des Prairies, Canada*

#### *\*Correspondence:*

*Markus Zedler, Department Clinical Psychiatry, Social Psychiatry, and Psychotherapy, Hannover Medical School, Carl-Neuberg-Str., 1 (OE 7110), 30625 Hannover, Germany e-mail: zedler.markus@ mh-hannover.de*

There is increasing evidence from case reports that synesthesia is more common in individuals with autism spectrum conditions (ASC). Further, genes related to synesthesia have also been found to be linked to ASC and, similar to synaesthetes, individuals with ASC show altered brain connectivity and unusual brain activation during sensory processing. However, up to now a systematic investigation of whether synesthesia is more common in ASC patients is missing. The aim of the current pilot study was to test this hypothesis by investigating a group of patients diagnosed with Asperger Syndrome (AS) using questionnaires and standard consistency tests in order to classify them as grapheme-color synaesthetes. The results indicate that there are indeed many more grapheme-color synaesthetes among AS patients. This finding is discussed in relation to different theories regarding the development of synesthesia as well as altered sensory processing in autism.

**Keywords: Asperger syndrome, autism, synesthesia, prevalence, development**

## **INTRODUCTION**

#### **SYNESTHESIA**

Synesthesia is a non-pathological phenomenon in which specific sensory stimuli (e.g., a sound) or concepts (e.g., time units or numbers) lead automatically to additional, internally generated sensations (e.g., colors, textures or shapes). A stimulus leading to synaesthetic sensations is termed "inducer," while the internally generated synaesthetic sensation is termed 'concurrent' (Grossenbacher and Lovelace, 2001). The most investigated type of synesthesia is grapheme-color synesthesia (GCS) in which numbers and/or letters are perceived in specific colors, occurring in about 1.1–2.0% of the population (Simner et al., 2006). The consistency of the specific synaesthetic couplings, i.e., that a certain letter always triggers a sensation of the same color, has been defined as one of the key features of synesthesia (Cytowic, 1995) and it has been shown to persist over long time periods (Simner and Logie, 2007). Although the view that synaesthetic coupling is always consistent over time has been questioned recently (Simner, 2012), consistency tests (Baron-Cohen et al., 1987; Eagleman et al., 2007) are still used as gold standard to distinguish synaesthetes from non-synaesthetes. A standardized online test battery (http://www*.*synesthete*.*org) has been developed for the classification of GCS, as well as other types of synesthesia (e.g., auditoryvisual synesthesia, spatial-sequence synesthesia, etc.). However, up to now the test for GCS is the most commonly used and its potential to differentiate synaesthetes from non-synaesthetes has been validated (Eagleman et al., 2007). Another key characteristic of synesthesia is the automaticity of consistent couplings, which can be tested with a speeded consistency test where inducers are presented either in a synaesthetically congruent or incongruent color (Eagleman et al., 2007). The idea of this test is that, as synesthesia occurs automatically, synaesthetes should be able to quickly distinguish synaesthetically congruent from incongruent trials. Indeed, synaesthetes have been shown to respond much more accurately in this test than controls.

As synesthesia runs in families, it is likely to have a genetic component: about 40–50% of synaesthetes report a first-degree relative who is also a synaesthete (Baron-Cohen et al., 1996; Barnett et al., 2008a,b). Further, specific gene loci have been identified as being related to this phenomenon (Asher et al., 2009; Tomson et al., 2011). However, as different types of synesthesia can occur within the same family, and as the synaesthetic experiences are highly individual for each synaesthete, it has been suggested that a genetic predisposition to develop synesthesia exists in general, but that the development of individual couplings is influenced by environmental factors (Barnett et al., 2008a,b).

The neuronal mechanisms underlying synesthesia still remain to be clarified. In GCS, there is evidence of the involvement of visual, parietal, and frontal brain areas (Rouw et al., 2011), whereas involvement of the parietal cortex has also been found in sequence-space (Tang et al., 2008) and (language-unrelated) auditory-visual synesthesia (Neufeld et al., 2012a).

#### **INDICATIONS FOR A RELATIONSHIP BETWEEN AUTISM SPECTRUM CONDITIONS (ASC) AND SYNESTHESIA**

Together with his co-workers, Simon Baron-Cohen published a case study on a remarkable man with synesthesia, Asperger Syndrome (AS) and savantism (Bor et al., 2007). They suggested

*<sup>2</sup> Department of Clinical Psychiatry, Social Psychiatry, and Psychotherapy, Hannover Medical School, Hannover, Germany*

that co-occurrence of AS and synesthesia might increase the likelihood of savantism. But there is also evidence for a linkage between ASC and synesthesia. While synesthesia is characterized by additional, internally generated sensations and in relation to that also often by unusual brain activation in sensory cortex regions (Rouw et al., 2011), altered sensory processing and sensory symptoms are also commonly found in individuals with ASC, e.g., in the visual (Simmons, 2009) and auditory domain (O'Connor, 2012; Samson et al., 2006). According to a recent review, sensory hypo- and hyper responsiveness are more frequent in ASC than in other developmental disabilities and atypical neural activity following sensory stimulation has been found in ASC patients even at the level of the primary sensory cortex in the auditory, tactile, and visual domain (Marco et al., 2011). It has been demonstrated that auditory stimuli can trigger responses in both auditory and nearby visual brain regions in autistic individuals (Kemner et al., 1995). Similarly, co-activation of the color processing area in the fusiform gyrus has been found in GCS (Hubbard, 2007) in response to the presentation of colorless graphemes. Further, increased connectivity has been detected in the brains of individuals with autism (Courchesne et al., 2005) as well as in synaesthetes (Rouw and Scholte, 2007; Jaencke et al., 2009; Haenggi et al., 2011). Findings showing altered brain anatomy in autistic individuals have led to the hypothesis that there is a developmental bias in ASC toward forming more short range connections, leading to hyper connectivity of local networks (Casanova and Trippe, 2009). Moreover, it has been suggested that cross-activation of adjacent brain areas are the mechanism underlying synesthesia (Hubbard and Ramachandran, 2005).

Besides these similarities, there is some evidence for a genetic linkage between ASC and synesthesia. Cytowic pointed out that about 15% of synaesthetes report having a first-degree relative with dyslexia, autism or attention deficit disorder (ADD) (Cytowic, 1995). A recent whole genome investigation with auditory-visual synaesthetes revealed a significant linkage of this type of synesthesia to genes which have previously been shown to be linked to autism (Asher et al., 2009; IMGSAC, 2001).

Finally, there is some anecdotal evidence for synesthesia in ASC patients (Harrison and Hare, 2004) and preliminary data based on self-report suggest that synesthesia is more common in individuals with AS (Johnson et al., 2011).

However, a systematic investigation into the relationship between AS and synesthesia using standardized tests is missing up to now. Here we tested a group of individuals diagnosed with ASC for GCS. We hypothesized that this type of synesthesia is more common in individuals with ASC.

## **METHODS**

#### **DIAGNOSTIC PROCESS**

DSM-IV criteria for AS in child- and adulthood (A.P.A., 1994) were thoroughly explored by a self-developed semi-structured interview ("Diagnostic interview: AS in adulthood"). After a general medical anamnesis (somatic, psychiatric and social history, including childhood development) the second part specifically explored symptoms related to AS. This part contained the sections: social interaction and communication (e.g., empathic abilities, friendships and interest in peers); special interests (e.g., in specific objects/topics); stereotype behavior (e.g., rituals, reaction to disturbance of rituals); and other characteristics (e.g., clumsiness, increased sensitivity toward sensory stimuli). Each section addressed child- and adulthood separately. Additionally, eye contact, facial expressions, prosody, "mirroring" of affections, and clumsiness were observed during the interview. The duration of the interview was about 90 min. Diagnosis was completed with information from personal interviews, either by telephone or in written form, with observers in child- and/or adulthood, such as partners, friends, parents or siblings. In some cases, school reports completed anamnesis.

All patients were interviewed by the same experienced investigator. Diagnosis was only confirmed if all DSM-IV criteria were clearly fulfilled. Retrospective data on the development of speech were assessed. Two male patients were excluded from the study retrospectively, as the possibility of delayed speech onset could not be reliably excluded in these cases.

Additionally we used two self-rating scales to complement diagnosis: the autism-spectrum quotient (AQ) (Baron-Cohen et al., 2001) and the empathy quotient (EQ) (Baron-Cohen and Wheelwright, 2004). The AQ is an instrument for quantifying where an individual is placed on the continuum from typical to autistic, a high score indicating more pronounced autistic traits. The AQ consists of 50 items which are divided into five subscales: social skill, attention switching, attention to detail, communication, and imagination. The EQ is an instrument for estimating the empathic abilities of an individual and a high score indicates greater empathy. It consists of 40 items on empathy and 20 filler items. Baron-Cohen suggested a cut-off score of ≥32 points for the AQ (80% of patients with AS scored more than 32 points) and a cut-off score of ≤30 points for the EQ (81% of patients with AS scored less than 30 points). However, the score for 20% of Baron-Cohen's patients was outside these cut-offs. Therefore those patients whose score was outside the autistic range were also included, as long as they fulfilled the DSM-IV criteria for AS.

Every patient was examined for axis-I co-morbidity by using the German version of the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I) (Wittchen et al., 1997).

#### **PARTICIPANTS**

All patients diagnosed with AS between February 2008 and June 2011 in the Clinic for Psychiatry of the Hannover Medical School (*n* = 29, 8 women) received a written or oral invitation to participate in the study, for which a small amount of monetary compensation was offered. The study was described as involving computer tests and questionnaires. Nothing was mentioned about synesthesia before participation to avoid a recruiting bias. In all, 21 patients (5 women, mean age = 37*.*9 ± 11*.*3 years) gave informed consent to participate in the study and completed the tests, as well as the questionnaires. The study was approved by the local ethics committee.

#### **EXPERIMENTAL PROCEDURE**

Once a participant arrived, he received a short description of synesthesia. A few examples of the different types of synesthesia, including GCS, were described. Furthermore, it was explained that synesthesia does not mean associations (e.g., associating the word "love" with the color red, or metaphorical associations, such as "being angry" = "seeing red"). Most of the participants claimed that they never heard of synesthesia before. After the introduction, participants were asked whether they experience color sensations when seeing numbers or letters. Participants then performed a consistency test for GCS (Eagleman et al., 2007), an offline version of the Synesthesia Test Battery (http://www*.* synesthete*.*org) on a PC. During this test, the numbers from 0 to 9 and the 26 letters of the alphabet were presented in black ink on white background on a computer screen. Each stimulus was presented separately and three times in a randomized order, so that there were 108 stimuli in total. Subjects were instructed to choose the color which they thought best fitted the presented grapheme by moving a cross-hair cursor on to a color matrix (see **Figure 1A**). As there was no time pressure to choose the colors in this test, the completion time varied (from about 20 min to 1 h).

After the test, participants completed a short questionnaire, a German version of the questionnaire used in the prevalence investigation by Simner et al., consisting of six questions (see supplementary material) which were rated on a six-point Likert scale to assess each participant's subjective view concerning consistency and vividness of potentially synaesthetic sensations experienced during the consistency test (Simner et al., 2006). Participants were asked whether their answers to these questions would be the same for numbers and letters and, if that was not the case, to fill in the questionnaire twice, once for numbers and once for letters. Responses ranged from "strongly disagree" ("trifft gar nicht zu") to "strongly agree" ("trifft voll zu") and were coded from 0 to 5. Scores between 0 and 30 could be achieved, with higher numbers indicating synesthesia (the highest score per question, 5, was assigned to "strongly agree" for questions 1, 3, 5, and 6 and to "strongly disagree" for questions 2 and 4). As a group of 20 known grapheme-color synaesthetes scored on average 26.4 in this questionnaire (*SD* = 4*.*64), the synaesthetic range has been defined as a score between 17 and 30 (lower bound = two standard deviations below mean score) (Simner et al., 2006).

In order to prevent false classification of participants using memorizing strategies to give consistent responses like synaesthetes, the consistency test was followed by a speeded consistency test. The latter is designed to test for the criterion of automaticity and is based on the assumption that the possibility of giving correct responses from memory is ruled out if the response has to be given quickly. (Eagleman et al., 2007). In this task, participants see a colored grapheme presented on the screen for 1 s. In 50% of trials, the color is congruent with the synaesthetic color reported by the participant, in the other 50% of trials the color is incongruent, thus different from that color. In each trial, participants are instructed to report as quickly as possible by pressing a button whether the color of the letter is congruent with their synaesthetic color or not. The test consists of 108 trials and takes approximately 10 min.

**FIGURE 1 | Graphical surface of the consistency test and examples of consistency test results for different patients. (A)** Participants selected a color for each item presented on the screen by moving a cross-hair cursor over a color matrix. The majority of patients reported not perceiving colors when seeing numbers or letters and made inconsistent color choices as in **(B)**. Accordingly, those patients have rather high scores in the consistency

test (*>*1.0). This is consistent with their reports and their low scores in the six-point questionnaire (*<*17). Three patients made consistent color choices (score *<*1.0) for letters and numbers **(C)** or numbers only **(D)** which was consistent with their reports and six-point questionnaire scores (those patients claiming to perceive synesthesia differently for numbers and letters completed the questionnaire twice, separately for numbers and letters).

Participants were only classified as synaesthetes if they fulfilled all three criteria: (1) subjective experience of synesthesia (assessed by self-report and short questionnaire), (2) consistency (indicated by a consistency score lower than 1.0 for those items for which synesthesia was reported) and (3) automaticity of synaesthetic sensations (indicated by an accuracy level of at least 80% in the speeded consistency test).

At the end, an extensive interview, based on a synesthesia questionnaire, was conducted to obtain additional information about the subjective experience of GCS, as well as other synesthesia types, and to collect some data on the patients (age, sex, education, etc.). One part of the synesthesia questionnaire was a list of possible possible inducers and concurrents, as in the questionnaire by Simner et al. (2006). Participants were asked to indicate any of synesthesia they thought they may have by drawing lines on the questionnaire between listed inducers and concurrents. They were further asked to name any additional type of synesthesia involving inducers or concurrents which were not on the list, or to modify items to describe the sensation in a better way. All in all, the whole investigation took approximately 2 h per subject.

## **DATA ANALYSIS**

#### *Consistency test*

As GCS is characterized by stable grapheme-color mapping, color consistency was calculated for the color choices per grapheme made during the three runs. The color variation was calculated from the geometric distance of the RGB (red green blue) values chosen for each item within the three trials. A consistency score was then calculated as the mean geometric distance per item over all N items (Eagleman et al., 2007). N was 36 – X (*X* = the number of items for which the "no color" button was chosen). Synaesthetes usually achieve scores smaller than 1.0, indicating greater consistency, while controls usually achieve higher scores (on average 2.0) indicating less consistent color choices.

#### *Speeded consistency test*

The percentage of correct responses in all 108 trials, as well as the mean response time, was then calculated. Synaesthetes have been shown to give 94% correct responses on average, while control subjects have an average of 67% correct responses (Eagleman et al., 2007). Therefore synaesthetes can be expected to respond correctly to at least 80% of the trials in this test.

#### *Statistical procedure*

For each participant, consistency score, short questionnaire score, percentage of correct responses and reaction times in the speeded consistency test were calculated. Patients were classified as synaesthetes if they (1) reported experiencing synesthesia before testing, (2) reached a consistency score *<*1.0 and an accuracy of *>*80% in the speeded consistency test. In this test, reaction times of participants classified as synaesthetes and those classified as nonsynaesthetes were compared to make sure that the former did not achieve higher accuracy by taking more time for their responses. For synaesthetes reporting synesthesia for both letters and numbers, mean reaction time for all graphemes was used while for those reporting synesthesia for numbers only, mean reaction time for numbers only was used. These reaction times were compared to (1) mean reaction times for all graphemes and (2) mean reaction times for numbers only in classified non-synaesthetes using two-sample t-tests. The same comparisons were made for consistency scores and percentage of correct responses in the speeded consistency test.

The percentage of patients classified as synaesthetes in the whole group included in the study, as well as in the group of patients tested, was calculated. To exclude the possibility that the percentage of synaesthetes found in the current study might be influenced by a recruiting bias (patients with synesthesia might be more likely to participate in a study), we used the percentage of synaesthetes found in the total sample of patients included, conservatively assuming that the eight patients who could not be tested would have been classified as non-synaesthetes. A twosided 95% Wald confidence interval was calculated for the percentage of synaesthetes in the whole sample, which was then compared with prevalence calculations of GCS in the general population. The prevalence of grapheme-color synaesthetes has been found to be 1.1% in museum visitors and 1.4% in students, when only those synaesthetes who perceived colors for both numbers and letters were counted. When those participants who perceived colors for numbers only were also counted as synaesthetes, Simner et al. found 2.0% grapheme-color synaesthetes in the university sample. As we also included both types of grapheme-color synaesthetes here, that is, those who perceive colors for letters and numbers and those perceiving colors for numbers only, we used the 2.0% as reference for our study. If GCS is more prevalent in AS patients, we would expect the 95% Wald confidence interval of our estimated percentage to range above 2.0%.

Furthermore a power calculation with nQuery7.0 based on the exact test for a single proportion was performed to investigate whether the sample size was appropriate for the comparison.

## **RESULTS**

## **PATIENTS REPORTING GCS**

Twenty-nine patients were included in the study, whereby 21 were tested. Out of those, six (two females) reported perceiving colors induced by letters and numbers or by numbers only. One of these six cases (patient 11) did not achieve a consistency score smaller than 1.0 (1.36 for all items, 1.27 for numbers). He also had only a marginal questionnaire score of 16 and described his sensations "more like feelings" than visual sensations and some numbers had "two colors at the same time" for him. It is not clear whether this is the reason for the relative inconsistency of his color choices during the test. He was therefore classified as non-synaesthete. The five remaining patients who reported perceiving synesthesia reached scores within the synaesthetic range in both the consistency test and speeded consistency test, either with regard to both letters and numbers or to numbers only (according to the type of synesthesia they reported). Out of these, one patient (patient 17) achieved a score of only 16 in the questionnaire which is just below the cut-off value of 17. He had said previous to the testing procedure that he might perceive colors for numbers and letters but that this happens more on a subconscious level. Even after completing the test he claimed that he was not sure if he really "sees" the colors, although it seemed quite clear to him which colors he should choose. He was also uncertain whether the color sensations occur automatically and whether they have always been the same, which is likely to be the reason why he responded rather conservatively to some questions in the six-point questionnaire. He further claimed to have difficulties concentrating and understanding the questions of the six-point questionnaire. As he achieved a clearly synaesthetic score of 0.75 in the consistency test and accuracy of 92.59% in the speeded consistency test (RT = 0.67 s), indicating that his color associations for graphemes are consistent as well as automatic, he was classified as a synaesthete. The four remaining patients scored clearly within the synaesthetic range in the questionnaire (21–26), as well as in the tests (see **Table 1**). In addition, all five patients classified as synaesthetes reported experiencing types of synesthesia other than GCS (for details see **Table 2**).

#### **PATIENTS NOT REPORTING GCS**

There was one patient (patient 15) who claimed to be unsure as to whether he perceives colors induced by graphemes; he had a score of 15 in the six-point questionnaire (which is within the non-synaesthetic range, although relatively close to the cutoff value of 17), but as his test scores were clearly within the non-synaesthetic range (consistency score = 1.83, speeded consistency = 76.9% correct) he was classified as a non-synaesthete. Among the remaining patients, no others claimed to perceive colors when seeing numbers or letters and none of them reached a score higher than 17 (0–14) in the six-point questionnaire, or a score ≤1.0 in the consistency test (see **Table 1**, for a test example see **Figure 1B**).

For the 16 patients who were classified as non-synaesthetes, the consistency score was between 1.19 and 2.59 (mean = 1.99; *SD* = 0*.*44) and the accuracy in the speeded consistency test was between 28.70and 79.63% (mean = 63.52%; *SD* = 13*.*90%). The mean questionnaire score was 8*.*44 ± 4*.*99.

When looking at RGB values for numbers only, the 16 patients classified as non-synaesthetes reached consistency scores between 0.92 and 3.29 (mean = 1.73; *SD* = 0*.*43) and an accuracy in the speeded consistency test between 33.33 and 83.33% (mean = 67,35%; *SD* = 15*.*03%).

**Table 1 | Demographic data, questionnaire score and consistency test results of all participants.**


*Prior to the tests, participants were asked whether they associate colors with numbers (N) or letters (L); they performed a consistency test, as well as a short questionnaire regarding their color associations during the test, and a speeded consistency test. In the questionnaire, synaesthetes usually have a higher score (>17), while in the consistency test lower scores indicate higher consistency which synaesthetes usually achieve (<1.0). In the speeded consistency test synaesthetes usually respond very accurately (on average 94%) compared to controls (on average 67%). Scores lying within the synaesthetic range are marked in red. Note that patient 13 was excluded from the study retrospectively, as the possibility that he had delayed speech onset could not reliably be excluded. GCS, grapheme-color synesthesia; m, male; f, female; AD(H)D, attention deficit (Hyperactivity) disorder.*

#### **Table 2 | Additional synesthesia types reported by patients classified as grapheme-color synaesthetes.**


**Table 2 | Continued**


*All patients classified as grapheme-color synaesthetes reported synesthesia types other than GCS, which were assessed by a questionnaire containing a list of possible inducers and concurrents.*

*f, female; m, male; OLP, ordinal linguistic personification.*

#### **COMPARING PATIENTS CLASSIFIED AS GC-SYNAESTHETES WITH THOSE CLASSIFIED AS NON-SYNAESTHETES**

The consistency scores, percentages of correct responses and reaction times in the speeded consistency test (both for all graphemes and numbers only) for the two groups were compared (see **Figure 2**). The distributions of all metrics were found to be not significantly different from normal distribution (Kolmogorov-Smirnov-Test) and therefore independent-sample t-tests were used for comparisons. Patients classified as synaesthetes were significantly more consistent in their color choices in the consistency test [*t(*20*)* = 5*.*79; *p <* 0*.*001] and in the speeded consistency test, regardless of whether the percentages of correct responses for the synaesthetes (letters and numbers or numbers only, depending on synesthesia type) were compared to the values for all graphemes [*t(*19*)* = 4*.*19; *p <* 0*.*001) or numbers only [*t(*19*)* = 3*.*47; *p <* 0*.*001) in the non-synaesthete group. However the reaction times in the speeded consistency test were not significantly different between groups when comparing the synaesthetes' values to non-synaesthetes' values for all graphemes [*t(*19*)* = 0*.*44; *p* = 0*.*67) or numbers only [*t(*19*)* = 0*.*24; *p* = 0*.*81). There was no significant correlation between response time and accuracy (*R* = 0*.*23, *p* = 0*.*31).This strongly suggests that those patients classified as synaesthetes did not achieve higher consistency by memorizing responses.

#### **STATISTICAL ANALYSIS**

All in all, six patients reported GCS and, of these, it was possible to verify five as synaesthetes: two perceiving letters and numbers equally in color, two perceiving only numbers in color and one perceiving strong colors induced by numbers and comparatively weak color associations induced by letters. These patients made up 23.8% of all patients tested and 17.2% of all patients included in the study (see **Figure 3**). For the more conservative

(GC-synaesthetes: mean =; SD =, non-synaesthetes: mean = 1.99; SD =),

estimation of 17.2%, the 95% Wald confidence interval extends from 3.5% to 31.0%, exceeding the prevalence of 1.1–2.0% found in the general population (Simner et al., 2006). Therefore the rate of synaesthetes was significantly higher in the patient group than in the general population. For the sample of tested patients the 95% Wald confidence interval extends from 5.6% to 42.0%.

The power calculation for the exact test for a single proportion with a two-sided type I error of 5% demonstrated that the sample size was large enough for the observed effect. The probability of proving a difference between 2 and 17.2% (the most conservative scenario) with 29 patients is 89%.

## **OTHER SYNESTHESIA TYPES**

Overall, 36 types of synesthesia other than GCS were reported by patients, regardless of whether they were classified as graphemecolor synaesthetes or not (see **Table 3**).

## **DISCUSSION**

significance.

#### **RELATION BETWEEN SYNESTHESIA AND ASC**

The results of the current pilot study indicate a much higher prevalence of GCS (between 3.5 and 31%) in patients with AS than in the general population. Although the sample size in this study is too small to make assumptions about the specific prevalence of synesthesia in individuals with ASC, there is strong evidence of increased prevalence in these patients, based not only on self-report, but on verification by standardized tests. Given that co-occurrence of synesthesia and ASC by chance is *per se* very unlikely (Bor et al., 2007), the current results clearly suggest a link between AS and synesthesia. This is in line with other reports of synesthesia in ASC patients (Harrison and Hare, 2004; Bor et al., 2007; Johnson et al., 2011) and with theoretical models showing that changes in sensory perception are linked to altered neuronal connectivity in ASC (Plaisted, 2001; Mottron et al., 2006, 2013). The possible reasons for such a linkage will be discussed below, taking these models into account.

There is considerable evidence for altered sensory perception in autism. For example, "sensory hypersensitivity" (Baron-Cohen

#### **Table 3 | List of reported synesthesia types apart from GCS .**


*With the assistance of the experimenter, all participants completed an extensive questionnaire where they were asked to report any other type of synesthesia they thought they might have by drawing lines between the inducers and concurrents listed. Patients were also encouraged to add inducers or concurrents to the list if they thought they had a type of synesthesia that was not represented. If requested, the experimenter explained any types of synesthesia which were unclear to the participant in more detail, giving some examples.*

*Reports from all study participants were considered here, regardless of whether they were classified as grapheme-color synaesthetes or not. Only the types of synesthesia reported are listed.*

*OLP, ordinal linguistic personification.*

et al., 2009) has been detected in different modalities (vision, audition and touch) in ASC individuals (Bonnel et al., 2003; Bertone et al., 2005; Blakemore et al., 2006; O'Riordan and Passetti, 2006; Tommerdahl et al., 2007; Heaton et al., 2008; Simmons, 2009). Further, increased brain activation in primary, as well as more associative, areas connected with visual processing has been found in a wide range of different cognitive tasks, as reported by a recent meta-analysis (Samson et al., 2012). Sensory hypersensitivity in the modality of the concurrent has also been detected in synaesthetes (Banissy et al., 2009) and enhanced visual evoked potentials unrelated to synaesthetic sensation have been found in synaesthetes as well (Barnett et al., 2008a,b). Therefore both phenomena are likely to be related to altered low-level perception.

The differences in low-level perception in autism have been explained by different models. An enhanced discrimination model proposes that hyperfunctioning of low-level perception, together with a deficit in recognizing similarities between stimuli or situations on the perceptual and attentional level, causes a lack of generalization (Plaisted et al., 1998; Plaisted, 2001). Similarly, a model of enhanced perceptual functioning in autism (EPF model) suggests that the main differences in perceptual processing between autistic and non-autistic individuals are determined by more locally oriented low-level sensory perception (leading to enhanced lowlevel discrimination) and more autonomy of low-level processing, e.g., reduced top-down influence (Mottron et al., 2006). Furthermore, the authors propose that there is enhanced midlevel perception in more associative areas, but in a nonstrategic, bottom-up or parallel fashion. This model emphasizes that variations in brain organization are the cause of the detected cognitive differences and atypical brain activation patterns toward more low-level areas (e.g., more posterior visual areas) during higher order perceptual tasks. Enhanced interaction between locally close sensory areas has not only been suggested to be a mechanism of an altered perception in autism, but also in synesthesia. Here the model of crossactivation proposes increased communication between locally adjacent brain areas involved in processing of inducer and concurrent, which would be the grapheme processing area and the color area in the fusiform gyrus in the case of GCS (Hubbard and Ramachandran, 2005).

If hyperconnectivity between sensory brain regions is the cause of altered sensory perception in autism as well as synesthesia, it is possible that a mutation in specific genes related to the development of structural or functional connections in the brain might increase the likelihood of both phenomena. For example, pruning have been suggested to be the reason for both synesthesia (Maurer, 1993; Hubbard et al., 2005) and AS (Bor et al., 2007), and increased increased (functional and structural) brain connectivity has been detected in synaesthetes (Rouw and Scholte, 2007; Jaencke et al., 2009; Haenggi et al., 2011) as well as Asperger patients (Belmonte and Yurgelun-Todd, 2003; Courchesne et al., 2005; Turner et al., 2006). Support for common genetic causes comes from a genetic marker demonstrated to be most significantly related to auditory-visual synesthesia has also been shown to been shown to be associated with ASC (Asher et al., 2009).

Besides the evidence for enhanced low-level perception and connectivity in synesthesia and ASC, there is also evidence for the involvement of associative brain regions in both phenomena. Increased activation in associative cortex regions involved in higher order sensory processing has been found in ASC individuals in the visual domain (Samson et al., 2012), while increased functional connectivity between frontal areas (Noonan et al., 2009) and between posterior cingulate and medial temporal cortex (Monk et al., 2009) has been found using fMRI. In synesthesia, the parietal cortex especially has been found to be hyperactivated in different types of synesthesia (Tang et al., 2008; Rouw et al., 2011; Neufeld et al., 2012a) and this region has also been found to be more strongly connected to the sensory areas involved in inducer- and concurrent processing (van Leeuwen et al., 2011; Sinke et al., 2012; Neufeld et al., 2012b), supporting the idea of top-down modulation of sensory areas by this higher-order associative region. It has been suggested in the socalled *two-stage model* that a combination of both increased local connectivity between sensory brain reagions and modulation of these connections by higher-order areas may be a mechanism of synesthesia (Hubbard and Ramachandran, 2005; Hubbard, 2007). The evidence for both low-level and top-down mechanisms in synesthesia implies that, while some components of synaesthetic perception might be "hard-wired," others might be more flexible and cognitive in nature. In line with this, it has been proposed that synesthesia has genetic, as well as developmental, components; for example, the likelihood of developing synesthesia seems to be related to genetic components, whilst the specific synesthesia type and the specific inducer-concurrent inducer-concurrent pairings are developed highly individually in every synaesthete (Rouw et al., 2011).

Therefore it is likely that synesthesia does involve building associations early in life. Further, we are currently unable to distinguish between synesthesia and associations, as the latter can also occur consistently and automatically if they are overlearned rather than built spontaneously. While originally synesthesia was often described as a "merging of the senses" and the term 'synesthesia' itself underlines its perceptual character, researchers point out nowadays that inducers are often conceptual rather than sensory in nature (Eagleman, 2012; Juergens and Nikolic, 2012; ´ Simner, 2012). It has been suggested recently that synesthesia might help to fill a "semantic vacuum" (which arises, for example, when a child learns letters) with meaning (Nikolic, personal ´ communication). Evidence for this theory comes from the fact that synesthesia is much more common for abstract inducers, such as letters, numbers or days (Day, 2005; Simner, 2012), and that for these inducers the context – and therefore the semantic content – determines the synaesthetic color rather than perceptual features (Juergens and Nikolic, 2012 ´ ). Therefore, synaesthetic experiences might make abstract information more concrete by acting as concrete labels and in that way make it more memorizable (Rothen et al., 2012). Synesthesia might, therefore, be one possible solution to the semantic vacuum problem, used preferentially – but not exclusively – by individuals suffering from deficits in abstract/conceptual thinking. It has been found that individuals with ASC differ from typical individuals regarding conceptual processing by being biased toward local instead of global processing, leading to a theory of 'weak central coherence', or a weakened drive to detect meaning by looking at the "big picture" (Happé and Frith, 2006). Difficulties in generalizing information might evoke a tendency in ASC patients to label otherwise abstract concepts with a concrete sensory experience, e.g., a color. On the other hand, enhanced memory abilities as found in individuals with ASC and Savant syndrome (Treffert, 2009) can be explained a more concrete way of processing information for these individuals. Besides the highly concrete representation of abstract concepts in savants, which has been reported anecdotally, (Murray, 2010), theoretical models showing the development of exceptional abilities related to autism point to the parallels with synesthesia (Murray, 2010; Rothen et al., 2012; Mottron et al., 2013). Specifically, the relation to synesthesia has been discussed in great detail in the context of veridical mapping, the coupling of homolog elements of recurrent isomorphic patterns, which has been proposed as an extension to the above-mentioned EPF model (Mottron et al., 2009). Veridical mapping includes, but is not restricted to, the strategic use of "if p, then q" rules (Mottron et al., 2013). The enhanced tendency to detect these rules within a system has been referred to as hypersystemizing, which has been suggested to be related to autism (Baron-Cohen et al., 2009). Interestingly, those domains which are most common as inducer categories (numbers, language, calendrical calculation, music) in synesthesia are (overlearned) linguistic sequences (Simner, 2012; Juergens and Nikolic, 2012 ´ ) which are also domains that are highly "systemizable" (Baron-Cohen et al., 2009). Further, the majority of synaesthetic inducers, as well as concurrents, consist of a series of homogeneous, meaningfully ordered elements (Mottron et al., 2013). In that sense, synesthesia can be regarded as resulting from associations between corresponding members of two homologous series and, therefore, as a form of veridical mapping. If veridical mapping is enhanced in ASC, this would explain the greater tendency of those individuals to develop synesthesia.

In conclusion, shared genetic components leading to differences in brain anatomy (including local connectivity) and, in relation to that, altered cognitive mechanisms like increased veridical mapping and a greater tendency to concretize abstract information might make individuals with ASC more likely to develop synesthesia.

#### **LIMITATIONS OF THE STUDY**

The main weakness of this study is the relatively small subject number – especially compared to other prevalence investigations. However, we tried to avoid response biases (1) by including all patients diagnosed by a single institution in a certain time period and (2) by not mentioning synesthesia before commencing testing, in order to avoid response bias driven by motivation to participate (which might be enhanced in individuals who believe that they are synaesthetes). Furthermore, the aim of the current study was not to evaluate the specific prevalence of synesthesia in AS patients, but to test the hypothesis that synesthesia is more common in this group than in the general population. Our results clearly support this hypothesis. On the other hand, they do not clarify whether AS is more common among synaesthetes. The question of the prevalence of AS in synaesthetes will have to be investigated in future studies.

Another critical point of this investigation is the diagnostic procedure. There is no standard for diagnosing AS according to the DSM-IV criteria in adulthood. The "Autism Diagnostic Interview – Revised" is often used, but this interview is based purely on information from the parents, with many questions concerning the patient's childhood. However, in adults that is often difficult, as patients do not wish to involve their parents in the diagnostic process or they are not available. Also, the retrospective recall of the childhood period may be imprecise. Therefore the diagnostic process for autism in adulthood is currently problematic, especially if no diagnosis has been made in childhood. We tried to minimize this problem by a thorough exploration of the DSM-IV criteria for child- and adulthood, by interviewing and observing the patients, and by supplementing this with information from third parties or, for example, from school records.

One might argue that instead of the prevalence estimation by Simner et al. a control sample of the same sample size as our patient group could have been used as a comparison. We believe that it is appropriate to compare our data with the prevalence estimation by Simner et al., as the latter is a very reliable source due to the large sample size. In addition, we used the same instructions and questionnaires, as well as a similarly designed synesthesia test. It should be mentioned that our sample of participants differs from the sample tested by Simner et al. regarding the sex ratio: while Simner et al. tested approximately the same number of men and women in the museum study (582 female and 608 male) and more women than men in the university study (327 female and 173 male), the majority of our participants were male (16 of 21). While Simner et al. found GCS to be equally common in men and women (1.1:0.9), a previous study suggested that it is about six times more common in females (Baron-Cohen et al., 1996). If the unbalanced sex ratio in our study affected the results, it might, therefore, be expected that an even higher percentage of synaesthetes would be found in a sample with a balanced sex ratio. Interestingly, two of the five patients classified as synaesthetes were women, although only five women were tested in total.

Given that AS is believed to be far more common in boys than in girls (8:1) (Remschmidt and Kamp-Becker, 2007), one might find the male/female ratio of 3.5:1 (2.75:1 in the whole sample) in this study surprising. However, there is more recent evidence that the proportion of AS actually varies less between males and females (on average 4.6:1, but varying in different states of the U.S.A. from 2.7:1 to 7.2:1) (Centers of Disease Control and Prevention, 2012). Lying within the range found in the United States, the male/female ratio in this study is therefore quite representative.

Further work is needed to examine the relationship between AS and synesthesia by investigating a larger group of Asperger patients and the possible reasons for the linkage or similarities between the two conditions. In addition, it would also be of great interest to learn whether the prevalence of ASC, or the occurrence of autistic traits (e.g., assessed by AQ and EQ), is higher among synaesthetes.

## **ACKNOWLEDGMENTS**

Christopher Sinke has been supported by the DFG (SFB 936/A4).

#### **REFERENCES**


adults with autism. *Brain Res.* 1154, 116–123. doi: 10.1016/j.brainres.2007. 04.032


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 July 2013; accepted: 21 November 2013; published online: 09 December 2013.*

*Citation: Neufeld J, Roy M, Zapf A, Sinke C, Emrich HM, Prox-Vagedes V, Dillo W and Zedler M (2013) Is synesthesia more common in patients with Asperger syndrome? Front. Hum. Neurosci. 7:847. doi: 10.3389/fnhum.2013.00847*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2013 Neufeld, Roy, Zapf, Sinke, Emrich, Prox-Vagedes, Dillo and Zedler. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Serotonergic hyperactivity as a potential factor in developmental, acquired and drug-induced synesthesia

## *Berit Brogaard\**

*Department of Philosophy and Center for Neurodynamics, University of Missouri, St. Louis, MO, USA*

#### *Edited by:*

*Beat Meier, University of Bern, Switzerland*

#### *Reviewed by:*

*Devin Terhune, University of Oxford, United Kingdom David Brang, Northwestern University, USA*

#### *\*Correspondence:*

*Berit Brogaard, Department of Philosophy and Center for Neurodynamics, University of Missouri, St. Louis, MO 63121, USA e-mail: brogaardb@gmail.com*

Though synesthesia research has seen a huge growth in recent decades, and tremendous progress has been made in terms of understanding the mechanism and cause of synesthesia, we are still left mostly in the dark when it comes to the mechanistic commonalities (if any) among developmental, acquired and drug-induced synesthesia. We know that many forms of synesthesia involve aberrant structural or functional brain connectivity. Proposed mechanisms include direct projection and disinhibited feedback mechanisms, in which information from two otherwise structurally or functionally separate brain regions mix. We also know that synesthesia sometimes runs in families. However, it is unclear what causes its onset. Studies of psychedelic drugs, such as psilocybin, LSD and mescaline, reveal that exposure to these drugs can induce synesthesia. One neurotransmitter suspected to be central to the perceptual changes is serotonin. Excessive serotonin in the brain may cause many of the characteristics of psychedelic intoxication. Excessive serotonin levels may also play a role in synesthesia acquired after brain injury. In brain injury sudden cell death floods local brain regions with serotonin and glutamate. This neurotransmitter flooding could perhaps result in unusual feature binding. Finally, developmental synesthesia that occurs in individuals with autism may be a result of alterations in the serotonergic system, leading to a blockage of regular gating mechanisms. I conclude on these grounds that one commonality among at least some cases of acquired, developmental and drug-induced synesthesia may be the presence of excessive levels of serotonin, which increases the excitability and connectedness of sensory brain regions.

**Keywords: acquired synesthesia, autism spectrum disorder, drug-induced synaesthesia, left-hemisphere injuries, multisensory perception, savant syndrome, serotonin hypothesis, traumatic brain injury**

## **INTRODUCTION**

Synesthesia is an extraordinary way of perceiving the world, involving experiences of connections between seemingly unrelated sensations, images or thoughts (Baron-Cohen et al., 1987; Cytowic, 1989; Rich and Mattingley, 2002; Sagiv and Ward, 2006; Brogaard, 2012). For example, seeing the number 7 may lead to an experience of navy blue, hearing the word "bliss" may flood the mouth with the flavor of bread soaked in tomato soup and hearing the key of C# minor may elicit a bright purple spiral radiating from the center of the visual field. The trigger of the experience is called "the inducer," whereas the additional experience to which it gives rise is called"the concurrent"(Grossenbacher and Lovelace, 2001). In visual synesthesia, the concurrent may be projected out into space and experienced as located in the visual scene outside the subject's mind, or it may be merely imagistically or semantically associated with the inducer (Dixon et al., 2004).

The two key characteristics of synesthesia regardless of whether it is of the projector or associator type is that it involves an aberrant binding of features from different sensory or cognitive streams that are associated with atypical conscious experiences or thoughts and that these experiences or thoughts are automatic, that is, synesthetes cannot suppress the association between an inducer and its concurrent. Other characteristics of the condition are specific to the different forms. According to Grossenbacher and Lovelace (2001), there are three different types of synesthesia:


Developmental synesthesia, the most common type, is a form of the condition that has persisted since birth or early childhood and that remains relatively stable and systematic over time: each inducer has a highly specific concurrent. (Baron-Cohen et al., 1987; Mattingley et al., 2001; Simner et al., 2006). It also tends to run in families (Baron-Cohen et al., 1996). For the most common forms of developmental synesthesia, the Synesthesia Battery, an automated online test, allows for rigorous testing of both the tightness of the synesthetic association and its stability and systematicity over time (www.synaesthete.org; Eagleman et al., 2007).

Acquired synesthesia is a form of the condition that emerges after brain injury or disease or artificial technologies like sensory substitution (Ward and Wright, 2012). It has been reported following stroke (Ro et al., 2007; Beauchamp and Ro, 2008; Thomas-Anterion et al., 2010; Schott, 2012), traumatic brain injury (Brogaard et al., 2012; Brogaard and Marlow, 2013), neuropathology involving the optic nerve and/or chiasm (Jacobs et al.,

"fnhum-07-00657" — 2013/10/17 — 21:48 — page 1 — #1

1981; Armel and Ramachandran, 1999; Afra et al., 2009), seizures (Jacome and Gumnit, 1979), migraine (Alstadhaug and Benjaminsen, 2010), post-hypnotic suggestion (Cohen Kadosh et al., 2009) and sensory substitution (Ward and Wright, 2012). Audiovisual synesthesia has been reported to be the most common acquired type (Afra et al., 2009). Like developmental synesthesia, acquired synesthesia tends to be automatic and systematic over time, though in some cases it only persists for a limited time period (Jacome and Gumnit, 1979; Lessell and Cohen, 1979; Afra et al., 2009). Experientally, acquired synesthesia may be indistinguishable from developmental synesthesia, though it is sometimes less inducer-specific, that is, the same concurrent may have several different inducers (Brogaard et al., 2012). Cases have also been reported in which the acquired experience is simpler than the developmental counterpart, often similar to light flashes (phosphenes) or pure color experiences (Afra et al., 2009).

Drug-induced synesthesia is a blending of sensory or cognitive streams that is experienced during exposure to a hallucinogen (psilocybin, LSD, mescaline, peyote; Shanon, 2002; Friedrichs, 2009; Sinke et al., 2012). Unlike the developmental and acquired varieties, the drug-induced form is usually limited to the most intense phases of intoxication, though in some cases it continues for weeks or months after exposure to the drug (Abraham, 1983; Ffytche, 2007). Experientally, drug-induced synesthesia can vary from simple color experiences to complex, surrealistic landscapes consisting of, for example, oddly shaped objects with multicolored contours or images with ornamental or kaleidoscopic compositions (Hintzen and Passie, 2010).

Though synesthesia research has seen a huge growth in recent decades, and tremendous progress has been made in terms of understanding the mechanism and cause of the condition, we are still left mostly in the dark when it comes to the mechanistic commonalities (if any) among developmental, acquired and drug-induced synesthesia. It is widely believed that most forms of the condition involve functional or structural aberrant brain connectivity. The proposed mechanisms include direct or indirect projection through increased structural connectivity (Ramachandran and Hubbard, 2001a,b; Hubbard et al., 2005; Rouw and Scholte, 2007; Jancke et al., 2009; Hanggi et al., 2011; Zamm et al., 2013), functionally driven disinhibited-feedback mechanisms (Grossenbacher and Lovelace, 2001; Dixon et al., 2006; Esterman et al., 2006; Neufeld et al., 2012), and mixed models (Hubbard, 2007; Ward, 2013). However, it is unclear what causes the onset of the condition and whether the different types of synesthesia have different causes.

One proposal by Brang and Ramachandran (2007) suggests serotonin (5-HT) as a causal factor. Their specific suggestion is that "serotonin S2a receptors are the 'synesthesia receptors' in the brain" (p. 903). In support of this hypothesis they list four pieces of evidence: (i) LSD produces synesthesia by selectively activating serotonin 5-HT2A receptors. (ii) Prozac (fluoxetine), a selective serotonin reuptake inhibitor that increases 5-HT1 receptor activity thereby inhibiting 5-HT2A, blocked synesthesia in two subjects. (iii) the anxiolytic drug Wellbutrin (bupropion), which presumably inhibits 5-HT2A receptor activity, temporarily abolished synesthesia in one subject, (iv) melatonin, a brain hormone derived from serotonin that can disinhibit 5-HT2A receptor activity, temporarily induced grapheme-color synesthesia in a subject with number-form synesthesia.

Thus, Brang and Ramachandran's (2007) suggestion is that serotonin may be functionally implicated in generating synesthetic experience through 5-HT2A receptor activity. In the formulation of the hypothesis Brang and Ramachandran (2007) do not specify which of the 5-HT2A receptors in the brain cause synesthesia, whether the serotonin receptors that are"synesthesia receptors"are inhibitory or excitatory, whether serotonin could be structurally implicated in causing synesthesia through altered structural connectivity during brain development and whether the serotonin hypothesis may also explain cases of acquired synesthesia. My aim in this paper is to develop the serotonin hypothesis to tentatively answer these questions by looking at a wider range of evidence.

More specifically, my proposal is that excessive extracellular serotonin (5-HT) can be a trigger of persistent or transient synesthesia in all the groups through *excitatory* mechanisms. Though serotonin traditionally has been considered an inhibitory neurotransmitter, more recent evidence suggests a more complex picture according to which serotonin can function both as an inhibitory and an excitatory neurotransmitter. For example, serotonin helps reduce fear processing in the amygdala via GABA modulation but it exerts an excitatory effect on cortical brain activity when it binds to 5-HT2A serotonin receptors on layer V pyramidal neurons (Barkai and Hasselmo, 1994; Aghajanian and Marek, 1999).

As indicated by Brang and Ramachandran (2007), the suggestion that serotonergic activity may be a trigger of synesthesia has the greatest degree of evidential backing in the case of druginduced synesthesia. It has been shown in several studies that psychedelic hallucinogens that function primarily as serotonin agonists, such as psilocybin, LSD and mescaline, often induce transient, often auditory-visual synesthesia, presumably through an alteration of functional brain connectivity. Though not all serotonin agonists elicit synesthetic experience, it is widely agreed that the mechanism of action for the class of serotonergic hallucinogens is through binding of serotonin to the 5-HT2A serotonin receptor (Presti and Nichols, 2004; González-Maeso et al., 2007).

Excessive serotonin levels may also play a role in synesthesia acquired after brain injury. Research has shown that necrosis following tissue damage leads to local neurotransmitter flooding caused by an excessive release of serotonin and glutamate (Busto et al., 1997; Hinzman et al., 2010), a phenomenon that can have long-range consequences even when the injury is minor. This flooding appears to lead to increased functional or structural interconnectedness among different brain regions in some individuals, and in some cases this increased connectivity may be a cause of synesthesia.

Developmental synesthesia has been reported as a condition in autism-spectrum disorders (Ornitz et al., 1978; Grandin, 1995; Kemner et al., 1995; Harrison and Hare, 2004; Asher et al., 2009; Baron-Cohen et al., 2009), and is believed to be more frequently occurring in individuals with autism compared to the general population (Cytowic, 1989). It is suggested below that synesthesia accompanying autism may be a serotonergic condition, resulting

"fnhum-07-00657" — 2013/10/17 — 21:48 — page 2 — #2

from excessive serotonin levels in early childhood that could possibly lead to decreased extracellular levels of serotonin in one hemisphere and compensatory increased levels in the other hemisphere. One piece of evidence for this comes from PET scans of people with high-functioning autism which have revealed that in the majority of cases serotonin synthesis is suppressed in the left hemisphere and increased in the right hemisphere, though in some cases it is reversed (Chandana et al., 2005). Recent genetic studies furthermore suggest a genetic link between autism spectrum disorder and developmental synesthesia in non-autistic individuals (Asher et al., 2009). Whether there is a mechanistic link between autism and cases of developmental synesthesia in non-autistic individuals remains to be established.

In what follows I will provide detailed evidence for the hypothesis that one commonality among at least some cases of acquired, developmental and drug-induced synesthesia is the presence of excessive levels of serotonin or serotonin-agonists, which increase the excitability and connectedness of sensory brain regions through 5-HT2A receptors in cortical neurons.

## **ACQUIRED SYNESTHESIA**

The acquired form of synesthesia usually emerges subsequent to traumatic brain injury or neuropathologic insult to the brain (Beauchamp and Ro, 2008; Afra et al., 2009; Brogaard et al., 2012). Several studies have hypothesized that these acquired synesthesias occur from plasticity of the sensory systems resulting in increased connectivity (Ro et al., 2007; Beauchamp and Ro, 2008). This theory has additionally been studied in at least one case of acquired synesthesia (Ro et al., 2012). Ro et al. (2012) found that connections between the auditory and somatosensory cortices in healthy controls were strengthened in a subject with auditory-tactile synesthesia acquired after a right ventrolateral thalamic lesion that deprived her somatosensory cortex of normal somatosensory input.

It remains largely unknown how a brain lesion may give rise to the plastic changes that lead to the increased connectivity. One possibility is that it is caused by increased neurotransmitter activity in cortical regions adjacent to the affected site. This increase is believed to contribute to the pathophysiology and neurological dysfunction after traumatic brain injury (Busto et al., 1997; Bullock et al., 1998; Robertson et al., 2001; Werner and Engelhard, 2007; Hinzman et al., 2010). In traumatic brain injury tissue damage makes the cells shift to anaerobic glycolysis, resulting in an accumulation of lactic acid. The anaerobic metabolism cannot by itself maintain the energy levels, so ATPstores are depleted and the membrane ion-pumps, which depend on ATP, fail. This leads to membrane degradation of vascular and cellular structures and necrotic or programmed cell death (apoptosis), resulting in excessive release of excitatory neurotransmitters, particularly serotonin and glutamate (Busto et al., 1997; Hinzman et al., 2010). This excess in extracellular serotonin and glutamate availability affects neurons and astrocytes and results in over-stimulation of serotonin and glutamate receptors. It is believed that this over-stimulation effect can happen even in mild traumatic brain injury, as the release of neurotransmitters is vast even with minor lesions (Konrad et al., 2011; Perez-Polo et al., 2013). Other types of brain injury that have been reported to trigger synesthesia, such as stroke (Ro et al., 2007; Beauchamp and Ro, 2008; Afra et al., 2009; Thomas-Anterion et al., 2010; Schott, 2012), also cause neurotransmitter flooding after necrosis. It has been found, for example, that even a brief ischemic stroke may trigger complex biochemical events that lead to progressive apoptotic and necrotic neuronal cell death (Yuan, 2009).

The increased levels of serotonin and glutamate immediately following brain injury do not normally stay elevated for very long (Eysel et al.,1999). Eysel et al. (1999)induced 1.5 to 2 mm lesions in the striate cortex of cats using surface photocoagulation or ibotenic acid injections. Single cell measurements revealed that activity was decreased at the border of the lesion and increased in a ring around the lesion during the first days to weeks following the induction but it returned to normal after about a month (Eysel et al., 1999). Despite the fact that serotonin does not seem to stay elevated for more than one month following a brain lesion, there is suggestive evidence that the temporary elevation may suffice for creating long-lasting functional, and possibly also structural, changes (Giza and Prins, 2006; Konrad et al., 2011; Perez-Polo et al., 2013). It has furthermore been reported that initial increases in neurotransmitter levels following brain injury or disease may be followed by down-regulation of receptors in ipsilateral brain regions (Giza and Prins, 2006).

One hypothesis for how brain injury or disease may cause synesthesia, then, is that the initially elevated neurotransmitter levels down-regulate serotonin receptors in neural regions in the ipsilateral hemisphere, leading to decreased serotonin levels. Individuals who acquire synesthesia also sometimes develop autistic traits and savant-like abilities of the sort seen in 10 percent of autistic individuals (Baron-Cohen et al., 2007; Treffert, 2009; Brang and Ramachandran, 2011; Rogowska, 2011; Brogaard et al., 2012; Schott, 2012; Ward, 2013). Emerging savant skills have also been reported in many other cases of central nervous system injury or disease later in life (Fay, 1987; Dorman, 1991; Miller et al., 1998, 2000; Lythgoe et al., 2005; Tammet, 2006; Treffert, 2006, 2009; Snyder, 2009). As is known from studies of autistic savants, downregulation of the serotonergic system in one hemisphere may result in an upregulation of the serotonergic system in contralateral brain regions (DeLong, 1999; Takeuchi et al., 2012), which might explain the development of savant skill and synesthesia following brain injury (Treffert, 2009). A second hypothesis is that the elevation in serotonin and glutamate levels in the days or weeks after brain injury can trigger disinhibited feedback or a structural binding of features through serotonergic hyperactivity in sensory neurons or neurons in parietal cortex involved in mental imagery. The latter have been found to be plausible neural correlates in at least one case of acquired synesthesia (Brogaard et al., 2012).

The second hypothesis is more plausible than the first when synesthesia is acquired after brain injury. It would explain the increased ipsilateral connectivity found in at least one case of acquired synesthesia (Ro et al., 2012). It gains further support from the fact that the onset of synesthesia following brain injury has been reported to occur shortly after the injury rather than months later, which indicates that it is the initial neurotransmitter flooding that causes the onset. Furthermore, if the first

"fnhum-07-00657" — 2013/10/17 — 21:48 — page 3 — #3

hypothesis were correct, then the affected region would have to be crucially implicated if the down-regulation triggers an upregulation in other areas, an assumption that cannot be confirmed at the present time. The first hypothesis may have a greater degree of support for synesthesia acquired after frontotemporal dementia. Cell death in frontotemporal dementia does not normally occur via necrosis that leads to neurotransmitter flooding. However, it is well known that patients with this condition have deficits in serotonergic and dopaminergic signal-transmission. These deficits may account for some of the cognitive and behavioral impairments of the disease. As prefrontal areas are known to exert inhibitory control over other brain regions, decreased activity in these neural regions could lead to a disinhibitory enhancement of neural activity and connectivity in unaffected cortical regions (DeLong, 1999; Treffert, 2009), which could also explain why this type of dementia frequently is reported as a cause of savant skills (Miller et al., 2000). Regardless of whether the synesthesia is acquired after brain injury or dementia, neuroimaging might help reveal whether the unusual binding is functional or structural. For example, if it is functional, we should expect to find increased connectivity in functional connectivity analyses on fMRI data.

The above considerations raise the question of why only a small fraction of brain injury patients acquire synesthesia-like experiences. One possible answer is that individual differences originate in variance in the degree of plasticity of the subject's brain prior to the incident. Research on brain injury in children has shown that alterations in neurotransmission during the critical period when a brain region is most plastic can promote outgrowth of abnormal neural connections (Giza and Prins, 2006). Since it is likely that there are significant individual differences in the plasticity of the mature brain, it is possible that some subjects are more susceptible to the formation of new neural connections than others. Individual differences are no doubt also grounded in variance in the location of the injury. Synesthesia-like experiences may be much more likely to develop if sensory regions or neural areas implicated in mental imagery are affected.

Given the broad range of conditions that can trigger acquired synesthesia it is unlikely that there is a single mechanism underlying all cases. Some cases appear to be quite similar in persistence and phenomenology to well-known forms of developmental synesthesia and are likely to share a neurological basis with some of these varieties of synesthesia. For example, autistic savant Daniel Tammet describes in his book *Born on a Blue Day* that he acquired synesthesia after childhood seizures. Though his grapheme-color synesthesia appears unusually rich, it may have neurological underpinnings akin to more typical developmental forms. Other forms of acquired synesthesia appear to be transient and may well be a more direct product of excitatory neural activity, similar in many respects to synesthetic experience occurring under the influence of hallucinogens (see below). An example would be synesthesia experienced during occipital and temporal lobe seizures, which can lead to brief experiences quite similar to psychedelic experiences (Devinsky and Luciano, 1991; Cytowic, 1997; Sacks, 2012). Some forms of acquired synesthesia could be undergird by a different mechanism not typically found after drug intoxication or aberrant neural development. In cases in which synesthesia and savant syndrome are acquired in the same incident, the two conditions may well be triggered in similar ways. This could be the case for Tammet, who reports having synesthesia matching not only digits but also the product of digits. His synesthesia could possibly be an imagistic manifestation of number processing in the parietal cortex. One observation that supports this hypothesis is that the lack of increased activity in the visual cortex in response to synesthetic tasks found in an imaging study comparing brain activity in Tammet and controls (Bor et al., 2007). But more research needs to be done to settle these questions. In most cases of acquired synesthesia the research component has taken place several years after the onset of the condition. If we could perform the research (e.g., neuroimagining studies) closer to the onset, we might be able to determine whether serotonergic hyperactivity in sensory neurons is implicated in the sudden, unusual binding of features.

## **DRUG-INDUCED SYNESTHESIA**

Drug-induced synesthesia is a blending of perceptual or cognitive streams that emerges in subjects under the influence of psychedelic hallucinogens, psychoactive substances that alter perception, mood, and a variety of cognitive processes. A myriad of first-person reports indicate that synesthesia occurs during psychedelic intoxication. Some reports suggest an altered perception of the world that blends normally distinct senses. Reports of colored music are particularly frequent (Shanon, 2002, 2003; Sinke et al., 2012). In other cases external objects appear to the perceiver as having an unusual wealth of colors, textures and shapes that undergo rapid changes. Subjects report seeing melting windows, breathing walls and spiraling geometrical figures crawling over the surfaces of objects. Reflecting on a DMT session one subject, described by Cott and Rock (2008), reported that "The room erupted in incredible neon colors, and dissolving into the most elaborate incredibly detailed fractal patterns that I have ever seen." The authors characterize this as a hallucination, and it is admittedly difficult to distinguish synesthesia and hallucinations, particularly because some forms of synesthesia probably are best characterized as hallucinations (Sagiv et al., 2011). A crucial difference seems to be that hallucinations *proper* do not have a phenomenally apparent inducer, whereas synesthesia does. Another difference is that most forms of synesthesia are experienced as endogenous images or representations (i.e., associator synesthesia, Dixon et al., 2004). Yet another difference is that synesthetes know that the concurrent experience is not a veridical perception, whereas hallucinations tend to evoke the experience of the veridicality of the perception (Terhune and Cohen Kadosh, 2012).

It is by now fairly well established that two major classes of psychedelic hallucinogens, the indoleamines (e.g., LSD and psilocybin) and the phenethylamines (e.g., mescaline), are potent partial agonists at serotonin 5-HT1A/2A/2C receptors, with 5- HT2A receptor activation directly correlated with hallucinogenic activity (Glennon, 1990; Vollenweider et al., 1998; Nichols, 2004; Presti and Nichols, 2004; though see e.g., Previc, 2011 for a different perspective). Though the mechanism of action varies for different hallucinogens, it is believed that 5-HT2A receptor activation of cortical neurons is responsible for mediating the

"fnhum-07-00657" — 2013/10/17 — 21:48 — page 4 — #4

signaling pattern and behavioral response to hallucinogens (Presti and Nichols, 2004; González-Maeso et al., 2007). However, the activation of the cortical serotonergic system does not fully explain the perceptual effects of psychedelic drugs, as not all 5-HT2A agonists (or partial agonists) have an excitatory mechanism of action and not all 5-HT2A agonists have psychedelic effects (e.g., methysergide). So, this raises the question of what other factors need to be present for drug-induced hallucinations to occur.

A promising suggestion for how the hallucinatory effects occur is that hallucinogens activate layerV pyramidal neurons in the cortex, which engage in gating functions in communication between the cortex and subcortical brain regions (Barkai and Hasselmo, 1994). When a hallucinogen binds to the 5-HT2A receptor, this gives rise to an excitatory response (Scruggs et al., 2003; Nichols, 2004). Recent research suggests that co-transmission of glutamate and monoamines is a frequent occurrence in the central nervous system (Trudeau, 2004; Ciranna, 2006). The 5-HT2A receptors, specifically, have been found to increase glutamate release (Ceglia et al., 2004; Torres-Escalante et al., 2004). There is furthermore evidence suggesting that the hallucinogen psilocybin targets a cortical receptor complex that forms when the glutamate mGluR2 receptor interacts with the serotonin 5-HT2A receptor (González-Maeso et al., 2008). Increased release of glutamate in response to hallucinogen administration should enhance cortical metabolic activity, a finding that has been confirmed by Vollenweider et al. (1997). Perceptual changes have been found to correlate with increased metabolic activity in the frontomedial and frontolateral cortices, anterior cingulate, and temporomedial cortex (Vollenweider et al., 1997). There is also some evidence from EEG studies that activation of 5-HT2A receptors increases the excitability of cortical sensory networks by modulating alpha oscillations (8–12 Hz; Kometer et al., 2013). Brain waves in the alpha frequency range have been shown to regulate the excitability levels of cortical sensory networks through inhibition (Klimesch, 2011). The increased excitatory action of serotonin and glutamate in sensory regions might explain why hallucinogens mimic aspects of psychosis during intoxication (Aghajanian and Marek, 1999; Pralong et al., 2002; Nichols, 2004; González-Maeso et al., 2008).

The view that the hallucinatory effects of hallucinogens are primarily a result of enhanced neural activity may appear inconsistent with a recent fMRI study showing that psilocybin causes decreased activity in the ACC/medial prefrontal cortex and a significant decrease in the positive coupling between the medial prefrontal cortex and the posterior cingulate cortex, and that these findings were correlated with subjective effects (Carhart-Harris et al., 2012). As Lee and Roth (2012) point out, however, these results are consistent with the observation that 5-HT2A receptors are found on both layer V glutamatergic neurons and GABAergic interneurons. Furthermore, there is also a direct activation of GABAergic interneurons through the synapses of pyramidal cells onto the interneurons (Markram et al., 2004). So, a large excitatory response in a pyramidal neuron will lead to a large inhibitory response in the interneuron. Lee and Roth (2012) propose that the effects of psilocybin could be due to both excitatory (e.g., pyramidal) and inhibitory (e.g., GABAergic interneuronal) neuronal circuits and that it may be the effects on the inhibitory neuronal circuits that gave rise to the measured BOLD response (see **Figure 1**).

To my knowledge, no model of how hallucinogens trigger synesthesia has yet been proposed. One natural proposal would be that the mechanisms in developmental and drug-induced synesthesia are similar. There is, however, some reason to doubt this suggestion. The phenomenological differences between developmental and drug-induced synesthesia are quite striking (see Sinke et al., 2012 for a review). As experiences causally supervene on neurological processes, experiences that are significantly different in their phenomenology are bound to have significantly different neurological underpinnings. So, we should expect some differences in the underlying mechanisms.

One tentative suggestion is that drug-induced synesthesia, like hallucinations, originates in the hyperactivity of layer V pyramidal cells resulting from the binding of hallucinogens to 5-HT2A receptors in the cells' dendrites, which then increases local glutamate levels. Layer V pyramidal cells bind multisensory information through feedback loops that synchronize oscillatory neural responses (Guillery and Harting, 2003). In the visual and the auditory cortices layer V neurons form feedback loops with local neurons as well as neurons in the thalamus and prefrontal cortex. Projections to thalamus play a role in discriminating among incoming information and integrating information from different sensory channels, whereas projections to the prefrontal cortex play a role in higher-order processes and the generation of a conscious representation. In normal multisensory perception, low-level multisensory binding of incoming signals from visual and auditory channels occurs spontaneously in the auditory cortex via thalamocortical feedback loops, when the spatial and temporal attributes of incoming signals match (Schroeder

"fnhum-07-00657" — 2013/10/17 — 21:48 — page 5 — #5

and Foxe, 2005). Excessive excitatory activity in layer V pyramidal neurons, however, results in a destabilization of layer V projections to the thalamus through GABAergic neuronal circuits (Kim and McCormick, 1998; Markram et al., 2004). This has a number of consequences, such as decreased attentional discrimination among incoming stimuli, allowing more information to flood the sensory cortices, a loss of stimulus-specific inhibition, an resultant increase of random (or environmentally under-constrained) activity in the thalamus, and a disruption of low-level, spontaneous integration of multisensory stimuli on the basis of *actually* matching spatial and temporal attributes. The disruption of low-level integration mechanisms can result in incongruent experiences, such as hearing an object hit the floor prior to seeing it fall. Another result of this disruption of low-level integration may be a coupling of stimuli that do not belong together. A common synesthetic experience during hallucinogen intoxication is colored, geometrical grids, matrices or fractals induced by music (see Sinke et al., 2012 for a review). These types of visual experience also frequently occur without an inducer, probably as a result of random activity in the thalamus (Behrendt and Young, 2004; Sagiv et al., 2011). One possible mechanism for drug-induced synesthesia, then, is that the brain assumes that an experience that results from occipital processing of random thalamic activity matches auditory stimuli, leading to an unusual low-level binding in the auditory cortex (see **Figure 2**). As a result of this aberrant binding, the two inputs may be experienced as an inducer-concurrent pair, for example colored, geometrical music.

The proposed mechanism leads to the testable prediction that drug-induced synesthesia will not usually be systematic. In developmental cases sound-color synesthesia is systematic in the sense that very specific sounds normally trigger very specific colors. Given the aberrant binding proposed, however, it appears that drug-induced sound-color synesthesia will not tend to be systematic because random activity will be paired with available auditory information. So, we should expect that the same sound may have many different colors and shapes.

There are only few reports of drug-induced synesthesia that endures post-exposure, though there is evidence that other practices that can induce altered states of consciousness, such as meditation and posthypnotic suggestion, frequently cause enduring synesthesia (Walsh, 2005; Cohen Kadosh et al., 2009). The suggested mechanism may help explain why drug-induced synesthesia is seldom persistent over time. The hypothesis proposed is that occipitally processed random activity is coupled with available auditory information during drug-induction. The random activity does not persist in the same magnitude after drug exposure, which may explain why the synesthetic experiences and hallucinations tend to subside. But what further decreases the chance that the synesthesia persists would be the lack of a one-one mapping or even a many-one mapping from inducer to concurrent, which we might expect will make the unusual synesthetic binding less likely to be retained as a neural connection.

The proposed mechanism of hyperexcitability followed by disruption may appear to be inconsistent with a related finding for

"fnhum-07-00657" — 2013/10/17 — 21:48 — page 6 — #6

developmental synesthesia. It is believed that enhanced cortical excitability at an early developmental stage might contribute to atypical grapheme-color binding, even though it does not appear to be a direct cause of synesthesia at later stages (Asher et al., 2009; Tomson et al., 2011). A recent study reported data that were consistent with the idea that at later stages the hyperexcitability can hinder synesthesia by producing excess noise in the visual cortex and thereby reducing conscious awareness of the atypical binding of features (Terhune et al., 2011). However, I think that this suggestion can be reconciled with the proposed mechanism for drug-induced synesthesia. It is plausible that hyperexcitability may no longer play a functional role at later stages of developmental synesthesia (Terhune et al., 2011). Hallucinogenic serotonin agonists, however, appear to introduce a significant amount of hyperexcitability that can offset normal binding by disrupting low-level integration mechanisms. It is possible that some varieties of developmental synesthesia form in early childhood in a way similar to the way druginduced synesthesia is here hypothesized to form temporarily later in life, but there would be certain differences: developmental synesthesia is systematic so if synesthesia develops in childhood as a result of a disruption of low-level integration, it is not a result of random activity in the thalamus. Furthermore, the presence of the hyperexcitability during brain development must somehow make it more likely that the unusual binding persists over time than when the hyperexcitability is drug-induced.

A version of the proposed model also appears consistent with the findings of Cohen Kadosh et al. (2009). The team induced strong projector grapheme-color synesthesia using posthypnotic suggestion in a group of highly suggestible college students. In most cases the synesthesia was found to endure and to display the phenomenal characteristics of developmental grapheme-color synesthesia. Cohen Kadosh et al. (2009) provide evidence suggesting that the induced synesthesia resulted from posthypnotically triggered disinhibited feedback. However, their results seem equally consistent with a model that proposes disruption of lowlevel integration. One shortcoming of both mechanistic proposals for synesthesia induced post hypnosis is that they do not explain why the condition tends to endure much longer than drug-induced synesthesia.

A further interesting question is whether the above model could be adapted to explain acquired synesthesia in which necrosis leads to neurotransmitter flooding and elevated local serotonin levels. Though we do not have enough data on acquired synesthesia to make any firm conclusions at this point, it remains a possibility that some forms of developmental synesthesia are acquired in childhood in the same way as the persistent cases of acquired synesthesia, whereas temporarily acquired synesthesia is mechanistically akin to drug-induced synesthesia. If this is correct, then we should expect drug-induced synesthesia and temporarily acquired synesthesia to both lack systematicity, as they would result from pairing features with random information. We should furthermore expect persisting forms of acquired synesthesia to have the same patterns of permanence and systematicity as at least some forms of developmental synesthesia.

## **DEVELOPMENTAL SYNESTHESIA IN AUTISTIC AND NON-AUTISTIC INDIVIDUALS**

Though it is difficult to say whether alterations in the serotonergic system play a role in the development of the most common forms developmental synesthesia, such as week-color and graphemecolor, developmental synesthesia seen in autism spectrum disorder is plausibly triggered by unilateral changes in the serotonergic system (Milner, 1973; Chugani et al., 1997; DeLong, 1999). There is not yet any solid evidence that autism and synesthesia are mechanistically related but there appears to be a statistical correlation as well as a possible genetic connection between the two conditions: synesthesia as well as sensory and perceptual abnormalities, such as hyper-responsiveness to sensory stimulation, are often reported as significant symptoms in autism-spectrum disorders (Ornitz et al., 1978; Grandin, 1995; Kemner et al., 1995; Harrison and Hare, 2004; Asher et al., 2009; Baron-Cohen et al., 2009). Cytowic (1989) states that 15 percent of people with autism experience synesthesia, which would be significant compared to the 4.4 percent of the general population (Simner et al., 2006). However, a systematic population study of the correlation between autism and synesthesia has not yet been completed. About 10 percent of individuals with autism also have savant syndrome, and all three conditions have frequently been reported to occur together (Baron-Cohen et al., 2007; Bor et al., 2007; Rothen et al., 2012). Recently, a genetic link between synesthesia and autism was suggested (Asher et al., 2009). Despite the lack of solid evidence for a mechanistic connection between autism and synesthesia, the existing statistical and genetic correlations give us reason to explore how serotonin may be related to synesthesia in autistic individuals. I will look at the evidence for the serotonin hypothesis for autism and then propose some testable predictions about how serotonin may give rise to synesthesia in autistic individuals.

The evidence that serotonin plays a crucial role in autism is overwhelming. About 30 percent of autistic individuals have a 25 to 70 percent increase in blood levels of serotonin, also known as hyperserotonemia (Schain and Freedman, 1961; Chugani et al., 1997, 1999; Veenstra-VanderWeele et al., 2012). Hyperserotonemia has also been found to a similar degree in first-degree healthy relatives (Leboyer et al., 1999). As serotonin cannot normally cross the blood-brain barrier in adults, high blood levels of serotonin are not necessarily a good indicator of high extracellular serotonin in the brain. High blood levels of serotonin, however, may indicate brain levels of serotonin in young children, as the blood-brain barrier is not fully developed until the age of two. Higher rates of autism have also been found in children exposed in utero to drugs that increase 5-HT levels, such as cocaine (Kramer et al., 1994). The high levels of serotonin in young children can negatively affect the development of serotonin neurons through negative feedback. As serotonin neurons develop and the extracellular levels of the neurotransmitter increase, growth of serotonin neurons is normally curtailed through a negative feedback mechanism, leading to a loss of serotonin terminals (DeLong, 1999; Whitaker-Azmitia, 2005). This decrease in serotonin terminal development has also been found in animal studies administering monoamine oxidase A and B inhibitors or serotonin reuptake inhibitors during gestation (Whitaker-Azmitia et al., 1994).

"fnhum-07-00657" — 2013/10/17 — 21:48 — page 7 — #7

Several PET imaging studies have suggested that autism may be a lateralized syndrome, with decreased serotonin release from raphe terminals in one hemisphere (typically the left) and elevated serotonin release in the contralateral hemisphere (Chugani et al., 1997, 1999; Chandana et al., 2005). Significantly increased language impairment was found in subjects with decreased serotonin synthesis in the left hemisphere compared to individuals with right-hemisphere abnormalities and those without cortical asymmetry. Further evidence for the lateralization theory comes from studies indicating functional improvement with selective serotonin reuptake inhibitors (SSRIs), such fluoxetine (DeLong, 1999). SSRIs block serotonin transporters, preventing extracellular serotonin from being transported back into the cell. When children with autism are treated with SSRIs, the symptoms they share in common with individuals with major depressive disorder and anxiety disorders drastically improve. The children's range of interests broaden, their perceptual experiences normalize, their memory and cognitive functions improve and anxiety and phobias become less prominent. Depleting brain serotonin in a tryptophan depletion paradigm, on the other hand, results in a worsening of the mood-related symptoms of autism as well as behaviors such as whirling, flapping, rocking and pacing (McDougle et al., 1996), though it does not affect the other symptoms of autism.

One possible explanation of the asymmetry is that the early serotonin depletion in the dominant left hemisphere leads to overcompensation in the right hemisphere. It has been reported that a decrease in extracellular serotonin over time may lead to an excessive spread of thalamocortical axon branches, resulting in lower information-transmission and structural changes in affected cortical regions as well as underdeveloped long-range connections between different brain areas (DeLong, 1999; Mueller et al., 2013). When the decrease in serotonin is left-lateralized, this leads to a general hypoexcitability of the left hemisphere and a wider right parietooccipital region compared to individuals with mental retardation or miscellaneous neurological disorders (Hier et al., 1979). The reversal of the asymmetry may be the result of initial right-hemisphere dominance, as indicated by a higher incidence of left-handedness and improved language skills in autistic individuals with decreased serotonin synthesis in the right hemisphere (Chandana et al., 2005). These results may explain why savant syndrome occurs in ten percent of autistic individuals. The leading hypothesis is that savant syndrome is caused by a lesion or birth defect in one hemisphere that results in overcompensation by the other hemisphere (Pesenti et al., 2001; Snyder et al., 2003; Young et al., 2004; Treffert, 2009; Loui et al., 2011).

A hemispheric effect does not provide clear evidence for a role of serotonin in synesthesia, as not all individuals with autism have synesthesia. But the expected high frequency of synesthesia in autistic individuals together with the lateralization hypothesis point to the possibility that increased extracellular levels of serotonin in the autistic brain may be a causal influence on the genesis of synesthesia in individuals with this disorder. There appears to be two ways in which serotonin could be implicated in synesthesia in autistic individuals. One possibility is that the high serotonin levels in very young children with autism sometimes trigger altered multisensory processing. Another possibility is that compensatory high serotonin levels in the contralesional hemisphere sometimes cause unusual feature binding. If the former hypothesis is correct, then we should expect to find evidence of synesthesia in autistic individuals at a very young age.

There are multiple ways that elevated serotonin levels may lead to synesthesia in autistic individuals. If the onset of synesthesia in autism occurs at an early age, it could be the result of serotonintriggered hyperactivity in glutamatergic neurons in layer V and resulting destabilization of thalamic connections. If this hypothesis is correct for synesthesia in autistic individuals, then we should expect that the synesthetic connections can persist only in the form of tight memory connections after the subsequent loss of serotonin-terminals in large areas of the brain. Accordingly, on the assumption that projector synesthesia is not normally mnemonic, we should not expect to find a significant number of projector synesthetes among autistic individuals. Another way synesthetic connections could form would be through the early excessive formation of neural pathways. Persisting synesthesia in this case would require that the early formed structural connections could survive the extensive pruning that appears to take place when serotonin-terminals are lost. If the onset of the synesthesia does not occur at an early age, structural or functional synesthetic connections could still form in the spared right-hemisphere regions that are also believed to be responsible for the savant skills founds in 10 percent of autistic individuals. Structural connectivity mechanisms have been proposed for standard cases of grapheme-color synesthesia, suggesting unusual connectivity between the color area and the fusiform gyrus (Ramachandran and Hubbard, 2001a,b; Hubbard et al., 2005). Enhanced anatomical connectivity near the fusiform gyrus confirming this hypothesis has been reported for grapheme-color synesthesia (Rouw and Scholte, 2007; Jancke et al., 2009; Hanggi et al., 2011) and sound-color synesthesia (Zamm et al., 2013). Whether a structural connectivity mechanism also underlies synesthesia in autism could fairly easily be tested by using a DTI paradigm to look at whether there are similar patterns of localized hyper-corticalconnectivity in autistic individuals with synesthesia. If the onset of the synesthesia does not occur at an early age, there is also the possibility that it is the result of impaired higher-order multisensory integration associated with decreased functional and structural connectivity (Mueller et al., 2013). This would make developmental synesthesia in autism very different from other cases, which appear to be a result of increased structural or functional connectivity. There is, however, a fairly strong reason against this mechanism as explanatory of synesthesia in autistic individuals. Impaired multisensory integration in autism amounts to a failure to associate two high-level sensory input that neurotypical individuals would associate (Mueller et al., 2013), not a persistent success in associating two input that neurotypical individuals do not associate.

Whether a hyperactive serotonergic system can contribute to developmental synesthesia in non-autistic individuals is unknown. As autism is partly defined by sensory processing deficits, the emergence of synesthesia in autism could be a consequence of other sensory alterations and originate from a unique set of

"fnhum-07-00657" — 2013/10/17 — 21:48 — page 8 — #8

mechanisms distinct from those present in other cases of developmental synesthesia. But some data points indicate that serotonin may be mechanistically involved in producing synesthetic experience through altered functional connectivity. As Brang and Ramachandran (2007) observed, there is some reported pharmacological evidence suggesting that developmental synesthesia in non-autistic individuals could sometimes be a serotonergic condition. Pharmacological evidence for the serotonin hypothesis was also reported by Cytowic (1989). He describes a patient with life-long synesthesia who developed epilepsy as an adult and subsequently experienced less vivid synesthetic experiences when treated with the anti-epileptic drug Tegretol (carbamazepine), which is known to increase extracellular levels of serotonin (Cytowic, 1989, p. 174). The reports from Cytowic (1989) and Brang and Ramachandran (2007) may seem to provide evidence against serotonin triggering synesthesia via excitatory activity, as the increased serotonin levels apparently inhibited synesthesia. However, both fluoxetine and carbamazepine have been shown to significantly increase GABA and reduce glutamate levels, which would block the excitatory effects of serotonin in cortical areas (Kamal, 2010). So, these data suggest that serotonin may be functionally involved in generating synesthetic experience either through a disinhibited feedback mechanism or by making unusual structural binding available for conscious processing. The pharmacological evidence thus lends some support to a disinhibited feedback mechanism, which suggests that synesthesia is not a result of altered structural connectivity but arises from altered functional feedback connections. This type of mechanism has received prior support from psychophysical and neuromagining studies of non-autistic synesthetes (Grossenbacher and Lovelace, 2001; Dixon et al., 2006; Esterman et al., 2006; Neufeld et al., 2012). A recent imaging study of 14 auditory-visual non-autistic synesthetes, for example, found increased functional connectivity of the left inferior parietal cortex with the left primary auditory and right primary visual cortex (Neufeld et al., 2012), suggesting that the aberrant synesthetic binding takes place in parietal cortex. There is also suggestive evidence of mixed mechanisms. For example, some forms of grapheme-color synesthesia appear to involve enhanced visual memory associations with hyper-reinstantiation in the visual cortex (see Brogaard, 2013 and Brogaard et al., 2013 for reviews).

Drawing on evidence that the 5-HT2A receptor may be involved in generating synesthetic experience, Brang and Ramachandran (2007) suggest that synesthesia might occur from overexpression of the 5-HT2A receptor gene on chromosome 13. A whole-genome linkage scan and a family-linkage analysis in a sample of 43 multiplex families with auditory-visual synesthesia did not confirm this hypothesis (Asher et al., 2009). Instead the study suggested that synesthesia may be traceable to a region on chromosome 2 (2q24.1) that has been implicated in autism (Newbury et al., 2009), indicating that there may be genetic link between developmental synesthesia and autism.

Other evidence gives some credit to the hypothesis that synesthesia in non-autistic individuals could be related in terms of brain structure to autism and savant syndrome and hence that a particular brain structure may underlie all of these conditions. Population studies suggest that there may be a higher incidence of synesthesia among people with creative talent (Domino, 1989; Mulvenna and Walsh, 2005; Simner et al., 2006; Rothen and Meier, 2010). Conversely, some synesthetes appear to have greater cognitive and memory capacities specific to the concurrent of the individual's synesthesia compared to the general population (Yaro and Ward, 2007; Ward et al., 2008; Banissy et al., 2009; Rothen et al., 2012). The possible association between synesthesia and cognitive talent might suggest that synesthetes without autism have serotonin-induced hyperconnected neural networks without the down-regulated neural regions found in people with autism. Recent neuroimaging studies further point to enhanced functional connectivity or increased gray matter density in synesthesia. A recent functional MRI study found increased intrinsic network connectivity in 12 grapheme-color synesthetes that reflected the strength of their synesthetic experiences (Dovern et al., 2012). Wiess and Fink (2009) further reported greater gray matter volume in the left intraparietal sulcus and right fusiform gyrus in 18 synesthetes compared to 18 controls. Several DTI studies have confirmed increased white matter connections in the superior parietal cortex, right inferior temporal cortex and frontal regions in grapheme-color synesthesia (Rouw and Scholte, 2007; Jancke et al., 2009, though at more liberal thresholds; Hanggi et al., 2011) and sound-color synesthesia (Zamm et al., 2013). Although these lines of evidence do not provide evidence of a causal connection among developmental synesthesia, autism and savant syndrome, results are suggestive that local hyperconnectivity is a common feature of these conditions.

However, we do not yet have enough data to draw any firm conclusions about the role of serotonin in developmental synesthesia in non-autistic individuals or the connection between developmental synesthesia and autism. More systematic population studies as well as whole-genome linkage scans and family linkage analyses may be able to shed more light on this connection. The extent to which serotonin is mechanistically involved in developmental synesthesia could be tested more systematically in pharmacological studies using drugs known to inhibit 5-HT2A receptors, such as SSRIs (Brang and Ramachandran, 2007), and serotonin-agonists, such as cocaine, that are only reported to cause hallucinations with excessive use. If serotonin is functionally implicated in synesthesia, we should expect 5-HT2A inhibitors to block or reduce synesthesia and serotonin-agonists to augment the experiences. A negative result would rule out that serotonin is functionally involved in synesthesia but would leave open the possibility that the neurotransmitter is involved in the onset of developmental synesthesia by leading to an early change in structural connectivity.

## **CONCLUSION**

The primary aim here has been to defend the hypothesis that synesthesia is at least sometimes a hyperserotonergic condition that triggers synesthesia through excitatory neurotransmitter action. The main evidence in favor of this hypothesis can be summarized as follows:

First of all, brain injury that gives rise to acquired synesthesia leads to necrosis and excessive release of serotonin and glutamate.

"fnhum-07-00657" — 2013/10/17 — 21:48 — page 9 — #9

Though this increase in excitatory neurotransmitter activity leads to lower excitability in local areas through negativefeedback within weeks, decreased activity in affected neural regions could lead to a disinhibitory enhancement of neural activity and connectivity in unaffected cortical regions. Alternatively, the early increase in serotonin levels could lead to the formation of unusual feature binding.

Secondly, administration of certain hallucinogenic serotonin agonists (e.g., psilocybin) induces synesthesia. The mechanism underlying drug-induced synesthesia plausibly involves

## **REFERENCES**


serotonergic excitatory activity in layer V pyramidal neurons implicated in multisensory binding.

Thirdly, serotonin synthesis is typically increased unilaterally in individuals with autism, about 15 percent of which are believed to experience synesthesia compared to about 4 percent population-wide. In terms of a potential mechanism, one possibility is that serotonin causes aberrant structural binding in the spared neural regions that are also responsible for the savant skills found in 10 percent of autistic individuals.

*Essays on Synaesthesia*, O. Deroy and M. Nudds (Oxford: Oxford University Press), (in press).


in develop-mental trajectory and asymmetry of cortical serotonin synthesis in autism. *Int. J. Dev. Neurosci.* 23, 171–182 doi: 10.1016/j.ijdevneu.2004.08.002


"fnhum-07-00657" — 2013/10/17 — 21:48 — page 10 — #10


Bondoux, D., Tabuteau, F., et al. (1999). Whole blood serotonin and plasma beta-endorphin in autistic probands and their first-degree relatives. *Biol. Psychiatry* 45, 158– 163. doi: 10.1016/S0006-3223(97) 00532-5


"fnhum-07-00657" — 2013/10/17 — 21:48 — page 11 — #11


from synesthesia. *Prog. Brain Res.* 155, 259–271. doi: 10.1016/S0079- 6123(06)55015-0


*of an Autistic Savant*. New York: Free Press.


"fnhum-07-00657" — 2013/10/17 — 21:48 — page 12 — #12

psychopathology in the psilocybin model of psychosis. *Neuropsychopharmacology* 16, 357–372. doi: 10.1016/S0893-133X(96)00246-1


synaesthesia. *Neurosci. Biobehav. Rev.* doi: 10.1016/j.neubiorev.2012.07.007 [Epub ahead of print].


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 19 July 2013; accepted: 21 September 2013; published online: 21 October 2013.*

*Citation: Brogaard B (2013) Serotonergic hyperactivity as a potential factor in developmental, acquired and druginduced synesthesia. Front. Hum. Neurosci. 7:657. doi: 10.3389/fnhum.2013. 00657*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2013 Brogaard. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

"fnhum-07-00657" — 2013/10/17 — 21:48 — page 13 — #13

## Probing the neurochemical basis of synaesthesia using psychophysics

## **Devin B. Terhune\*, Seoho M. Song, Mihaela D. Duta and Roi Cohen Kadosh**

Department of Experimental Psychology, University of Oxford, Oxford, UK

#### **Edited by:**

Nicolas Rothen, University of Sussex, UK

#### **Reviewed by:**

Jean-Michel Hupe, CNRS - University Paul Sabatier, France Jamie Ward, University of Sussex, UK

#### **\*Correspondence:**

Devin B. Terhune, Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK

e-mail: devin.terhune@psy.ox.ac.uk

The neurochemical mechanisms that contribute to synaesthesia are poorly understood, but multiple models implicate serotonin and GABA in the development of this condition. Here we used psychophysical tasks to test the predictions that synaesthetes would display behavioral performance consistent with reduced GABA and elevated serotonin in primary visual cortex. Controls and synaesthetes completed the orientation-specific surround suppression (OSSS) and tilt-after effect (TAE) tasks, previously shown to relate to GABA and serotonin levels, respectively. Controls and synaesthetes did not differ in the performance parameter previously associated with GABA or in the magnitude of the TAE. However, synaesthetes did display lower contrast difference thresholds in the OSSS task than controls when no surround (NS) was present. These results are inconsistent with the hypothesized roles of GABA and serotonin in this condition, but provide preliminary evidence that synaesthetes exhibit enhanced contrast discrimination.

**Keywords: disinhibition, GABA, serotonin, synaesthesia**

## **INTRODUCTION**

Grapheme-color synaesthesia is an unusual neurological condition in which letters and numerals reliably elicit involuntary color experiences (for a review, see Ward, 2013). Individuals with this condition have been shown to display enhanced color discrimination (Yaro and Ward, 2007; Banissy et al., 2009, 2013) and color working memory (Terhune et al., 2013), which may relate to larger visual-evoked potentials for stimuli that bias the parvocellular visual pathway in this population (Barnett et al., 2008). The extents to which these results are reflective of broader differences in visual processing among synaesthetes remain unknown, but predictions regarding such differences can be derived from competing models of this condition. Two neurotransmitters, γ-aminobutyric acid (GABA) and serotonin (5-HT), have been proposed to contribute to synaesthesia by different models (Grossenbacher and Lovelace, 2001; Cohen Kadosh and Henik, 2007; Brang and Ramachandran, 2008; Brogaard, 2013). Both play fundamental, but differential, roles in visual processing and are strong candidates for exploring the neurochemistry of synaesthesia.

GABA is a major cortical inhibitory neurotransmitter, and GABAergic interneurons comprise approximately 1/6 of neurons in the cortex (Buzsáki et al., 2007). These cells subserve a range of cortical processes such as the segregation of competing neuronal assemblies, cortical maturation, and the shaping of network oscillatory patterns (Möhler, 2007; Buzsáki and Wang, 2012). Resting-state GABA levels in primary visual cortex, as measured by magnetic resonance spectroscopy (MRS; Stagg et al., 2011a; Puts and Edden, 2012), have been shown to covary with orientation discrimination (Edden et al., 2009) and orientation-specific surround suppression (OSSS; Yoon et al., 2010). Specifically, it has been observed that primary visual cortex GABA concentrations are negatively correlated with orientation detection thresholds (Edden et al., 2009) and positively associated with an orientation suppression ratio believed to index inhibition (Yoon et al., 2010). In addition, several studies have demonstrated that GABA<sup>A</sup> agonists attenuate visual processing along with its electrophysiological correlates (Giersch and Herzog, 2004; Watson et al., 2009; van Loon et al., 2012). It has been argued that disinhibition theories of synaesthesia (Grossenbacher and Lovelace, 2001; Cohen Kadosh and Henik, 2007), which propose that synaesthesia arises from disinhibited feedback from higher cortical areas, predict that synaesthesia is characterized by reduced GABA concentrations in synaesthesia-relevant regions (Hubbard et al., 2011; Specht, 2012). That is, the diminished cortical inhibition proposed to contribute to synaesthetic perception by these theories should be associated with lower GABA levels. A number of studies have provided direct or indirect evidence in support of disinhibition models (e.g., Cohen Kadosh et al., 2009; Terhune et al., 2011), but none have directly evaluated the prediction of reduced GABA in the synaesthetic brain.

Serotonin is a neurotransmitter that plays an instrumental role in a wide range of functions including visual processing. In particular, evidence from the psychopharmacological literature indicates that serotonergic pathways, especially those involving the S2a serotonin receptor, are implicated in visual aberrations produced by amphetaminergic compounds such as methylendioxymethamphetamine (MDMA; Brown et al., 2007; Dickson et al., 2009; White et al., 2013) or hallucinogens such as psilocybin (Marek and Aghajanian, 1998; Kometer et al., 2011, 2013). Perhaps unsurprisingly, serotonergic neurons have also been suggested to regulate visual-motor gating (Pum et al., 2008) as well as cross-modal/cortical sensory integration (Jitsuki et al., 2011; Takahashi, 2011). To date, two hypotheses have highlighted serotonin's role in the development of synaesthesia (Brang and Ramachandran, 2008; Brogaard, 2013). Brang and Ramachandran (2008) have argued that the S2a serotonin receptor may underlie hyperconnectivity between the fusiform gyrus and area V4, which has been predicted to play an instrumental role in the development of this condition (Hubbard, 2007; Hubbard et al., 2011). Brogaard (2013) similarly argues that elevated serotonin in the striate cortex may contribute to the development of synaesthesia and may underlie acquired and induced synaesthesias. Perhaps the best evidence to date for these models comes from a recent meta-analytic review, which shows that the induction of synaesthesia-like experiences in non-synaesthetes is most reliably observed with serotonin agonists, such as lysergic acid diethylamide (LSD), mescaline, and psilocybin (Luke and Terhune, 2013). There is also preliminary evidence that serotonin agonists enhance synaesthesia in congenital synaesthetes (Simpson and Mckellar, 1955; Luke et al., 2012). However, these studies suffer from a number of methodological limitations (Luke and Terhune, 2013) and only provide preliminary support for serotonin models. Although congenital and induced synaesthesias may not share the same neurochemical mechanisms, these studies do suggest that serotonin may be implicated in synaesthesia.

The aim of the present study was to test predictions derived from disinhibition and serotonin models of synaesthesia using proxy psychophysics measures of GABA and serotonin. Toward this end, controls and synaesthetes completed a task measuring OSSS (Xing and Heeger, 2001; Zenger-Landolt and Heeger, 2003; Yoon et al., 2009, 2010). Target identification is poorest when the surrounding region has an orientation parallel to the target and performance is facilitated by surround suppression (Chubb et al., 1989; Xing and Heeger, 2001; Yoon et al., 2009). The magnitude of this suppression effect has been long believed to be driven by inhibitory mechanisms and has recently been shown to correlate positively with GABA concentrations in primary visual cortex (Yoon et al., 2010). Insofar as disinhibition theories predict reduced GABA in synaesthesia, we tested the prediction that synaesthetes would display less surround suppression owing to a lower inhibition-derived sharpening of the stimulus orientation. In particular, we expected that synaesthetes would perform less optimally than controls where inhibitory fine-tuning processes would benefit the most such as when the parallel-oriented surround obscures the target.

Measurement of serotonin-related performance relied on a visual phenomenon known as the *tilt-after effect* (TAE; Paradiso et al., 1989; Masini et al., 1990; Murray et al., 2012). When participants are adapted to a single stimulus with a certain angular orientation, their perception of the following stimuli's angular orientation tends to be biased in the direction opposite that of the adaptive stimulus (Masini et al., 1990). This is due to the saturation of neurons that are specific to the orientation of the adaptive stimulus, which dampens the neurons' sensitivities to subsequent stimuli (He and Macleod, 2001; Murray et al., 2012). Decreasing cortical serotonin, such as through tryptophan depletion, hinders this inhibition, stretches the neurons' tuning bandwidth, and augments the magnitude of the TAE (Masini et al., 1990; Brown et al., 2007; Murray et al., 2012). If synaesthesia is characterized by elevated serotonin in primary visual cortex, synaesthetes should display an attenuated TAE.

## **METHODS**

## **PARTICIPANTS**

Sixteen controls (14 female, *M*Age = 23.1, *SD* = 4.3) and 15 grapheme-color synaesthetes (13 female; *M*Age = 25.1, *SD* = 4.8), all of whom were right-handed and had normal or corrected-tonormal vision, provided informed consent to participate in this study in accordance with approval from a local ethics committee. The two groups did not differ in gender distributions, Fisher's exact *p* = 1, age, *F* < 1.5, or years of (post-secondary) education (controls: 3.8 ± 1.9, synaesthetes, 4.5 ± 2.6), *F* < 1.

In order to compute consistency of grapheme-color associations, participants selected colors using a color picker for the numbers 0 through 9 in random order three times in a serial fashion (Eagleman et al., 2007; Rothen et al., 2013). On the second and third rounds, controls were instructed to try to select the same color that they previously selected for each respective grapheme. Synaesthetes displayed greater consistency (lower values reflect greater consistency) with a measure based on city block distances in RGB color space (Eagleman et al., 2007) (controls: 1.85 ± 0.22; synaesthetes: 0.58 ± 0.04; *H* = 13.81, *p* < 0.001, η 2 *<sup>p</sup>* = 0.51 [CIs: 0.23, 0.66]) and a measure based on Euclidean distances in CIELUV color space (Rothen et al., 2013) (controls: 143.91 ± 14.58; synaesthetes: 33.03 ± 15.06; *H* = 16.58, *p* < 0.001, η 2 *<sup>p</sup>* = 0.49 [CIs: 0.21, 0.65]).

## **MATERIALS**

## **Orientation-specific surround suppression (OSSS)**

The OSSS task measures the participant's ability to discriminate contrast differences between targets in variously oriented surrounds (Zenger-Landolt and Heeger, 2003). Stimuli were roundels that contained contrast reversing (4 Hz), gray-scale sinusoidal gratings with a spatial frequency of 2.2 cycles per degree (see **Figure 1A**). Each stimulus contained an *annulus* that was divided into eight sectors and the *surround* that was partitioned into central and outer regions. The central region extended from the center of the stimulus to the inner radius of the annulus (2.9◦ ). The outer region extended from the outer radius of the annulus (5.6◦ ) to an eccentricity of 9.2◦ . The contrast of both surround regions was kept at 100%. The central portion of the surround included a circle of radius 1.0◦ .

The OSSS task had three surround conditions: *no surround* (NS), *parallel surround* (PS), and *orthogonal surround* (OS). The PS and OS conditions contained gratings in the surround that were parallel or perpendicular to the annulus, respectively. 50% of the trials randomly included a target (a single randomly selected annular sector with a lower contrast relative to the remainder of the annulus sectors) and participants judged whether the target was present or absent. The contrast of the target ranged from 38% to 73% and was parametrically varied in steps of 5% depending on performance using a 3-up, 1-down adaptive staircase procedure. The contrast of the non-target annular regions was fixed at 75%. Stimuli were presented for 750 ms with an interstimulus interval of 1050 ms.

## **Tilt-after effect (TAE)**

This task provides a measure of the TAE (Paradiso et al., 1989; Masini et al., 1990). The stimuli consisted of two offset, 4-layered circular sectors (see **Figure 1B**). The right-hand sector had a larger diameter than the left, and collectively, the stimulus had a diameter of 9◦ . The right sector was tilted by a certain angle toward the left (*negative* tilt) or the right (*positive* tilt) of the vertical midline. There were three phases in this task: *pre-adaptation*, *adaptation*, and *post-adaptation*. In the pre-adaptation phase, participants were presented with stimuli that contained one of nine randomlyselected tilt angles (−4 ◦ to +4◦ with 1◦ intervals), and judged whether the tilt was negative (leftward) or positive (rightward). During the adaptation phase, participants were exposed to a stimulus with either a +15◦ or a +40◦ tilt angle for 90 s. Participants were instructed to scan over the stimulus without fixating on a single position in order to avoid after-images. In the postadaptation phase, participants judged the tilt direction of stimuli of thirteen possible tilt angles (−4 ◦ to +8◦ with 1◦ intervals). Stimuli were presented in 10-trial blocks that were interspersed with a 5 s adaptation interval. Stimuli in the pre- and postadaptation phases were presented for 100 ms with interstimulus intervals of 500 ms.

#### **PROCEDURE**

Participants were recruited for a research study on synaesthesia (synaesthetes) or visual perception (controls) through advertisements at the University of Oxford. Participants completed the tasks in counterbalanced order on a Dell P190ST monitor (resolution: 1280 × 1024; refresh rate: 60 Hz) at a distance of 50 cm. The OSSS was implemented using the Psychophysics Toolbox software<sup>1</sup> (Brainard, 1997; Kleiner et al., 2007) for MATLAB® (2012a, MathWorks Inc, Natwick MA), whereas stimulus presentation for the TAE task was implemented using E-Prime® (2.0, Psychology Software Tools Inc., Sharpsburg PA). Responses were recorded using a Cedrus® response pad (Cedrus Corporation, San Diego, CA). Participants completed a practice block of 20 trials in the OSSS task and one 150-trial block of the NS, OS, and PS conditions with the OS and PS block order counterbalanced. In the TAE task, participants completed 90 trials in the preadaptation phase and 26 10-trial blocks of each tilt angle in the post-adaptation phase with tilt angle conditions counterbalanced.

## **DATA ANALYSIS**

Data in the OSSS task were modeled using the Palamedes Toolbox (Prins and Kingdom, 2009) in MATLAB. The probabilities of a correct response to targets at each contrast level were fitted with a Weibull function defined by four parameters: threshold α, slope β, guess rate γ , and lapse rate λ. Threshold and slope were set as free parameters that were estimated using maximum likelihood estimation, whereas guess and lapse rates were fixed at 0 and 0.1, respectively. In addition to the loss of one synaesthete's data due to a technical error, nine participants' data displayed poor model fit (*p*Devs < 0.05; Kingdom and Prins, 2010) in one or more conditions. Model fit was substantially improved through the removal of a single outlier in seven participants (two controls, five synaesthetes), whereas the data of two participants (one control, one synaesthete) could not be improved and thus were excluded from the analyses (the principal results [including the Group differences] are the same if these participants are included). The analyses of the OSSS data therefore included 15 controls and 13 synaesthetes. Model fit did not differ as a function of Condition or Group, nor was there an interaction, *F*s < 1, *p*s > 0.34. The 79% threshold of the psychometric function (*contrast difference threshold*) at each condition was used as our principal dependent measure of interest (Yoon et al., 2010). Threshold values provide a measure of the discriminability of the stimulus with lower values (in the present study) reflecting superior discrimination. As a secondary measure of interest, we also analysed function slopes, which index the steepness of the psychometric function fit to the data. More negative slopes indicate greater steepness of the function and thus greater deterioration in performance from one contrast difference to the next.

The same procedure was applied to the data from the TAE task with the following differences. One synaesthete did not perform the task correctly and provided unusable data and a second synaesthete's data were lost due to a technical error. Preadaptation data in the 15◦ and 40◦ conditions did not differ and thus were combined. The probabilities of a positive (rightward) response at each tilt angle were fitted with a logistic function with the same parameter constraints as in the OSSS task. The intersection of the function and the 0.5 threshold was taken as the point of subjective equality (PSE), which corresponds to the tilt angle that is perceived to be approximately equally likely to be negative (leftward) and positive (rightward). Six participants' data displayed poor model fit (*p*Devs < 0.05); of these, model fit was

<sup>1</sup>http://psychtoolbox.org

substantially improved through the exclusion of a single outlier in three participants (two controls, one synaesthete), whereas the data of three participants (two controls, one synaesthete) could not be improved and thus were excluded from the analyses. Accordingly, the analysis of the TAE data included 14 controls and 12 synaesthetes. Model fit did not differ as a function of Condition or Group, nor was there an interaction, *F*s < 2.2, *p*s > 0.13. PSEs for the different conditions were used as the principal dependent measure and function slopes were included as a secondary measure of potential interest. The TAE was quantified as the difference between PSEs before and after adaptation for both the 15◦ and 40◦ conditions.

#### **STATISTICAL ANALYSES**

Statistical analyses of the data were conducted using SPSS® (21, IBM) and MATLAB. Outliers were detected using the adjusted boxplot rule (Pernet et al., 2013) and replaced using a nearest neighbor correction (nearest extreme value ±1). Data were analysed with analyses of variance (ANOVA) with Group as the between-groups factor and Surround type (NS v. OS v. PS; OSSS task), or Stimulus type (15◦ vs. 40◦ TAE task) as withingroups factors, depending on the analysis. We used unequal variance *t*-tests (Welch, 1947) when data violated the assumption of homogeneity of variance across groups. We applied the Greenhouse-Geisser correction when the assumption of sphericity was violated. Uncorrected *df*s are reported for the latter two analyses. To control for the possibility of false positives, a false discovery rate (FDR) correction (Benjamini et al., 2001) was applied to the entire set of *p*-values comprising analyses for both tasks; only corrected *p*-values are reported.

#### **RESULTS**

#### **ORIENTATION-SPECIFIC SURROUND SUPPRESSION (OSSS)**

Performance on the OSSS task is illustrated in **Figure 2A**. The analysis of contrast difference thresholds revealed main effects of Surround type, *F*(2,52) = 26.34, *p* = 0.013, η 2 *<sup>p</sup>* = 0.50 (CIs: 0.29, 0.63), reflecting a linear increase in thresholds across conditions, and Group, *F*(1,26) = 6.89, *p* = 0.039, η 2 *<sup>p</sup>* = 0.21 (CIs: 0.01, 0.44), reflecting lower contrast difference thresholds (better discrimination) among synaesthetes, but no interaction, *F*(2,52) = 1.09, *p* = 0.49, η 2 *<sup>p</sup>* = 0.04 (CIs: 0.00, 0.16). To determine the breadth of group differences across conditions, particularly since the PS condition requires greater inhibition, we conducted exploratory analyses comparing the groups in the three surround conditions. Synaesthetes displayed lower contrast difference thresholds in the NS, *F*(1,26) = 7.07, *p* = 0.039, η 2 *<sup>p</sup>* = 0.21 (CIs: 0.01, 0.44), and OS, *F*(1,26) = 10.64, *p* = 0.021, η 2 *<sup>p</sup>* = 0.29 (CIs: 0.04, 0.51), conditions, but not in the PS condition, *t*(26) = 1.55, *p* = 0.28, η 2 *<sup>p</sup>* = 0.08 (CIs: 0.00, 0.31). The former two effects remained significant when model fit in the respective condition was included as a covariate, NS: *F*(1,25) = 6.62, *p* = 0.041, η 2 *<sup>p</sup>* = 0.21 (CIs: 0.01, 0.44), OS: *F*(1,25) = 10.70, *p* = 0.021, η 2 *<sup>p</sup>* = 0.30 (CIs: 0.04, 0.52). These results suggest that synaesthetes display superior contrast discrimination than controls.

In the analysis of slopes, there were main effects of Surround Type, *F*(2,52) = 13.34, *p* = 0.013, η 2 *<sup>p</sup>* = 0.34 (CIs: 0.13, 0.49), reflecting a linear decrease in slopes across the three conditions, no effect of Group, *F*(1,26) = 2.24, *p* = 0.28, η 2 *<sup>p</sup>* = 0.08 (CIs: 0.00, 0.30), but a significant interaction, *F*(2,52) = 7.07, *p* = 0.039, η 2 *<sup>p</sup>* = 0.21 (CIs: 0.04, 0.37). Subsidiary analyses revealed that synaesthetes displayed steeper slopes in the OS condition than controls, *F*(1,26) = 9.73, *p* = 0.022, η 2 *<sup>p</sup>* = 0.27 (CIs: 0.03, 0.49), but that the two groups did not differ in the NS, *F*(1,26) = 0.06, *p* = 0.83, η 2 *<sup>p</sup>* < 0.01 (CIs: 0.00, 0.08), or in the PS, *F*(1,26) = 0.02, *p* = 0.89, η 2 *<sup>p</sup>* < 0.01 (CIs: 0.00, 0.03), conditions. The results indicate that synaesthetes exhibited steeper slopes in the OS condition than controls; this appears to have resulted from the fact that synaesthetes exhibited superior discrimination in this condition than controls at contrast differences of 22% and 17%, but thereafter exhibited a greater decline in discrimination relative to controls, resulting in numerically poorer performance at 12% contrast difference.

Insofar as controls and synaesthetes differed in both OS thresholds and slopes and these two variables were strong correlated, *r* = 0.70, *p* < 0.001, we performed two ANCOVAs repeating these analyses controlling for the other variable. Neither analysis was significant, threshold (controlling for slope): *F*(1,25) = 2.22, *p* = 0.28, η 2 *<sup>p</sup>* = 0.08 (CIs: 0.00, 0.31), slope (controlling for threshold): *F*(1,25) = 1.54, *p* = 0.38, η 2 *<sup>p</sup>* = 0.06 (CIs: 0.00, 0.28), indicating a clear interdependence between these two variables. In contrast, the Group difference in NS thresholds remained significant when controlling for slopes, *F*(1,25) = 7.79, *p* = 0.039, η 2 *<sup>p</sup>* = 0.24 (CIs: 0.02, 0.47).

The contrast thresholds for the OS and PS conditions were further evaluated by controlling for performance in the NS condition through the computation of ratios of contrast thresholds. These analyses were performed to examine performance in suppression conditions (OS and PS) controlling for baseline contrast discrimination. Synaesthetes displayed a lower OS/NS ratio, *t*(26) = 2.52, *p* = 0.044, η 2 *<sup>p</sup>* = 0.19 (CIs: 0.00, 0.43), but did not differ in the PS/NS ratio, *t*(26) = 0.46, *p* = 0.73, η 2 *<sup>p</sup>* = 0.01 (CIs: 0.00, 0.16). The former result suggests that synaesthetes display enhanced surround suppression, although this effect became non-significant when controlling for OS slopes, *F*(1,25) = 0.98, *p* = 0.33, η 2 *<sup>p</sup>* = 0.04 (CIs: 0.00, 0.24). The most crucial analysis concerned the PS/OS ratio, which has been associated with GABA concentrations in primary visual cortex (Yoon et al., 2010). If this relationship holds and synaesthetes display reduced GABA in this region, synaesthetes should exhibit a larger ratio. A non-significant Group effect, *F*(1,26) = 0.97, *p* = 0.49, η 2 *<sup>p</sup>* = 0.04 (CIs: 0.00, 0.24), including when controlling for OS slopes, *F*(1,26) = 0.03, *p* = 0.88, η 2 *<sup>p</sup>* < 0.01 (CIs: 0.00, 0.04), shows that this prediction was not supported. Cumulatively, these results suggest that, in contrast with the predictions of disinhibition models, synaesthetes do not display poorer inhibition than controls, but that synaesthetes display superior contrast discrimination when no surround is present.

#### **TILT-AFTER EFFECT (TAE)**

The results of the TAE are presented in **Figure 2B**. On the basis of the finding that lower V1 serotonin levels are associated with a more pronounced TAE (Murray et al., 2012), we expected that synaesthetes would display an attenuated TAE relative to

controls. First, the two groups did not exhibit differential PSEs in the pre-adaptation phase of the task, *F*(1,24) = 0.25, *p* = 0.73, η 2 *<sup>p</sup>* = 0.01 (CIs: 0.00, 0.19). The analysis of PSEs in the TAE task revealed a main effect of Stimulus type, *F*(1,24) = 9.07, *p* = 0.028, η 2 *<sup>p</sup>* = 0.27 (CIs: 0.03, 0.50), in which adaptation to 40◦ produced a positive tilt magnitude and adaptation to 15◦ resulted in a negative tilt, but there was neither a main effect of Group, *F*(1,24) = 0.49, *p* = 0.65, η 2 *<sup>p</sup>* = 0.02 (CIs: 0.00, 0.21), nor an interaction, *F*(1,24) = 0.14, *p* = 0.76, η 2 *<sup>p</sup>* < 0.01 (CIs: 0.00, 0.16). Contrary to earlier findings (Paradiso et al., 1989; Murray et al., 2012), the tilt magnitude at 15◦ suggested increased rightward responses following adaptation, yet this was unrelated to between-group differences. The analysis of slopes similarly found no pre-adaptation differences as a function of Group, *F*(1,24) = 0.38, *p* = 0.69, η 2 *<sup>p</sup>* = 0.02 (CIs: 0.00, 0.20), main effects of Stimulus type, *F*(1,24) = 0.91, *p* = 0.49, η 2 *<sup>p</sup>* = 0.04 (CIs: 0.00, 0.25), Group, *F*(1,24) = 0.24, *p* = 0.73, η 2 *<sup>p</sup>* = 0.01 (CIs: 0.00, 0.18), or an interaction, *F*(1,24) = 1.84, *p* = 0.33, η 2 *<sup>p</sup>* = 0.07 (CIs: 0.00, 0.30). Cumulatively, these results indicate that controls and synaesthetes do not differ in tilt perception or in the magnitude of the TAE.

#### **DISCUSSION**

This study tested predictions pertaining to altered neurochemistry in synaesthesia derived from disinhibition and serotonin models of this condition using proxy psychophysical measures. In contrast to both models, the two groups did not display differential response patterns on task measures previously associated with visual cortex GABA and serotonin levels. Exploratory analyses revealed that synaesthetes displayed lower contrast difference thresholds in one condition of an orientation task than controls, suggesting superior contrast discrimination. These results are somewhat equivocal with regard to their implications for the involvement of GABAergic disinhibition or elevated serotonin as chemical underpinnings of synaesthesia, but suggest that these neurochemicals are not altered in this condition.

In contrast to the results suggesting enhanced contrast discrimination in synaesthetes relative to controls (see below), the two groups did not differ in the ratio of contrast thresholds in the parallel and OS conditions (PS/OS ratio). The between-groups factor of Group explained ∼4% (CIs: 0.00, 0.24; controlling for slope: <0.01 [CIs: 0.00, 0.04]) of the variance in this ratio and therefore, when considering the sample sizes of controls and synaesthetes, renders it unlikely that this null result is due to low statistical power. The PS/OS ratio has previously been shown to correlate with GABA concentrations in primary visual cortex (Yoon et al., 2010) and thus was used to test the prediction, derived from disinhibition models (Cohen Kadosh and Henik, 2007; see also Hubbard et al., 2011), that synaesthetes would exhibit reduced GABA. These results suggest that synaesthetes do not display reduced inhibition in primary visual cortex, but caution should be exerted in the interpretation of these results for multiple reasons.

First, the relationship between GABA and the PS/OS ratio is correlational and this ratio may not provide a robust proxy measure of visual cortex GABA. That is, it may be the case that this measure does not constitute a sufficiently stringent test of the prediction of reduced GABA in synaesthesia. For example, one reason to doubt the link between GABA and PS/OS ratios in the Yoon et al. (2010) study is that patients with schizophrenia were medicated, which may represent an important confound in the measurement of GABA levels in this population (Kegeles et al., 2012). However, it is readily evident that the relationship between GABA and the PS/OS ratio in Yoon et al. (2010) is driven by controls, rather than schizophrenics. Moreover, a recent study found that when applied to the primary visual cortex, anodal transcranial direct current stimulation, previously shown to reduce cortical GABA (Stagg et al., 2009), attenuated surround suppression on a similar orientation discrimination task (Spiegel et al., 2012). This strengthens the claim that surround suppression is indeed related to individual differences in visual cortex GABA.

Second, the evidence for the PS/OS ratio as a proxy measure of GABA in striate cortex comes from a study showing that it correlates with GABA in primary visual cortex, as measured by MRS (Yoon et al., 2010). The available evidence suggests that MRS-derived GABA concentrations seem to reflect extrasynaptic GABA (GABA tone) (Stagg et al., 2011a,b). Accordingly, it is possible that disinhibition-specific differences associated with GABAergic interneurons contributing to the expression of synaesthesia may not be detectable with MRS-derived measures of cortical GABA concentrations. Rather, it may be more informative to focus on the density and distribution of various types of GABA receptors, which are the principle foundations upon which GABA exercises its influences on neuronal activity.

Finally, if we assume that the PS/OS ratio provides a reliable measure of visual cortex GABA, it is plausible that differences in GABAergic activity between controls and synaesthetes are actually restricted to fusiform gyrus or V4 or higher cortical areas such as parietal cortex (e.g., van Leeuwen et al., 2011), rather than primary visual cortex. Indeed, elsewhere we have argued that cortical hyperexcitability in primary visual cortex in synaesthetes does not play a causal role in the online experience of synaesthesia; rather, it may have only contributed to the expression of synaesthesia at an early developmental stage (Terhune et al., 2011). At present, we are unable to discriminate between these competing interpretations of the PS/OS results. Nevertheless, the results clearly do not support disinhibition theories, but the extent to which they are inconsistent with them is as yet unclear.

Controls and synaesthetes also completed the TAE task, which provided a measure of the magnitude of the TAE. The TAE is believed to be augmented in conditions characterized by low serotonin in primary visual cortex (Paradiso et al., 1989; Brown et al., 2007; Murray et al., 2012) and was used to test the prediction that synaesthetes would display a reduced TAE. In contrast with this prediction, the two groups did not differ in the magnitude of the TAE, suggesting that synaesthesia is not characterized by elevated serotonin in striate cortex. As was the case with the critical parameter in the OSSS task (PS/OS ratio), the magnitude of this effect was very small (∼2%; CIs: 0.00, 0.21) and thus, given our sample sizes, it is unlikely that this analysis was underpowered. As is the case with the OSSS task, there are a number of explanations for these null results. First, previous research used the TAE to identify serotonin deficiencies (Masini et al., 1990; Brown et al., 2007; Murray et al., 2012) and thus it could be argued that the TAE is not well suited to detect elevated serotonin levels. We failed to observe any clear evidence for ceiling effects on this task and thus this explanation seems unlikely. A second possibility is that the TAE is not a reliable measure of visual cortex serotonin levels. Given the number of studies linking serotonin with the TAE, we also find this unlikely. For example, one study found that acute tryptophan depletion augmented the magnitude of the TAE (Masini et al., 1990). Tryptophan is the physiological precursor to serotonin and has been shown to produce a temporary reduction in 5-HT (2) receptor binding (Yatham et al., 2001). Accordingly, we interpret the current results to suggest that controls and synaesthetes do not differ in V1 serotonin. If serotonin plays a decisive role in the occurrence of synaesthesia, the present results suggest that it is most likely in other (downstream) cortical regions, such as fusiform gyrus and V4.

Recent evidence has surfaced raising concerns about the brain region responsible for the TAE. In particular, it has been suggested that the TAE is driven by retinotopic as well as cortical mechanisms (Knapen et al., 2010; Mathot and Theeuwes, 2013). This possibility is controversial because a retina-based mechanism would be unable to account for differences in the TAE caused by ecstasy consumption (Brown et al., 2007; Murray et al., 2012) or by tryptophan depletion (Masini et al., 1990). The former studies arguably challenge the validity of the TAE as an indirect measure of cortical serotonin. However, the psychophysical paradigms used to demonstrate a retinotopic component to the TAE are only conceptually similar to the assays used in the present study (e.g., they employ structurally disparate stimuli). Hence, this limitation of the TAE does not severely undermine the TAE's capacity to represent cortical serotonin.

Further work is required to explore whether GABA and serotonin are implicated in synaesthesia. This may be achieved by using MRS to measure GABA in fusiform gyrus and V4 or through methods for modulating visual cortex serotonin (e.g., tryptophan depletion). The study of synaesthesia-like experiences following the intake of serotonin agonists has especially strong potential to inform our understanding of the role of this neurochemical in synaesthesia (Brang and Ramachandran, 2008; Brogaard, 2013; Luke and Terhune, 2013). There is also considerable potential that disinhibition and serotonin models can be integrated. For instance, it has been shown that elevated serotonin shifts the balance between excitation and inhibition in favor of excitation (Moreau et al., 2010). Accordingly, elevated serotonin could potentially give rise to a state of cortical hyperexcitability and attenuated inhibition (see also Terhune et al., 2011).

A novel finding of this study is that synaesthetes displayed superior contrast difference thresholds than controls. This effect was present both in the NS and OS conditions and thus is unlikely to reflect a difference in surround suppression, although only the former difference remained when controlling for condition slopes. Synaesthetes similarly exhibited steeper slopes in the OS condition, which reflect the steepness of the psychometric function fit to individual participants' data. Although synaesthetes outperformed controls at mid-range contrast differences (17 and 22%), their performance was more greatly taxed at lower contrast differences (12 and 7%); below we speculate as to why this might be. Insofar as OS thresholds and slopes were inter-dependent, the apparent superior contrast discrimination among synaesthetes in the OS condition should be interpreted with caution. Superior performance among synaesthetes in the NS condition is suggestive of superior contrast discrimination in this population, but further research is necessary to determine the replicability of this effect. The observed performance difference appears to converge nicely with those of Barnett et al. (2008) who observed that synaesthetes display enhanced visual-evoked potentials for stimuli that are preferentially processed by the parvocellular visual pathway but not those that bias the magnocellular pathway. Although the low spatial frequency of the stimuli used in the OSSS task (see Section Methods) biases magnocellular neurons (e.g., Derrington and Lennie, 1984), the observed group difference pertains to the detection of contrast differences between the target and the surrounding sectors of the annulus. That is, the target differed in contrast from the surrounding sections, but not in spatial frequency. The magnocellular pathway saturates at low levels of contrast (Merigan and Maunsell, 1993; Lee, 1996) whereas the parvocellular system is recruited for ∼10% contrast and greater (Tootell et al., 1988). Accordingly, it is significant that the group differences in the NS condition of the OSSS task, based on the absence of overlap of standard error bars (see **Figure 2A**), are only present at 12% and higher contrast differences. This suggests that contrast discrimination advantages among synaesthetes emerge at contrasts for which the parvocellular pathway is preferentially recruited; these results are therefore consistent with the proposal that synaesthetes exhibit enhanced responsiveness of the parvocellular system (Barnett et al., 2008; see also Rothen et al., 2012).

The observations that synaesthetes displayed selectively superior contrast discrimination, but did not differ in PS/OS ratios or the TAE (see below), are also significant for multiple reasons. First, as has been noted elsewhere (Gross et al., 2011; Radvansky et al., 2011; Terhune et al., 2013), the specificity of performance advantages reduces the likelihood that superior performance is driven by greater motivation among synaesthetes (see also Banissy et al., 2013). Second, it is noteworthy that synaesthetes no longer outperformed controls in the OSSS condition that most strongly taxes cortical inhibition (PS). It is plausible that superior contrast discrimination and deficient cortical inhibition come into conflict with one another in this condition and that elevated contrast discrimination among synaesthetes may mask inhibition differences. Further research is required to more precisely investigate this possibility.

The present results should be interpreted within the context of the limitations of this study. We have argued that the analyses showing null results pertaining to the psychophysical parameters putatively related to GABA and serotonin are unlikely to be underpowered given the small effect sizes. We have also corrected for multiple analyses to reduce the likelihood of reporting false positive results. However, as in any study with small samples, it is possible that the observed effects will disappear with larger sample sizes. It is plausible that the differential performance patterns of controls and synaesthetes are due to elevated motivation in the latter group rather than differences in contrast discrimination, but we have argued that this is unlikely. Finally, the fits of psychometric functions to behavioral data were poor for a subset of participants, who displayed poor performance in one or more contrast differences (OSSS task) or angles (TAE task). We were able to correct most instances of poor fit with outlier removal, but this might reduce the generalizability of our results. Taken together, these limitations warrant that the contrast discrimination difference between controls and synaesthetes be replicated before firm conclusions regarding enhanced contrast discrimination among synaesthetes can be advanced.

## **CONCLUSION**

This study contrasted controls and synaesthetes in two psychophysical measures in order to test predictions derived from disinhibition and serotonin models of synaesthesia. The two groups did not differ in the performance patterns predicted by these models, but synaesthetes displayed superior contrast discrimination than controls in the absence of surround stimuli. Although the tasks used do not constitute rigorous evidence against these models, they suggest that synaesthetes do not exhibit atypical GABA or serotonin levels in primary visual cortex.

## **ACKNOWLEDGMENTS**

This research was supported by the Cogito Foundation (Devin Blair Terhune). Devin Blair Terhune is supported by a Marie Skłodowska-Curie Intra-European Fellowship within the 7th European Community Framework Programme. Roi Cohen Kadosh is supported by the Wellcome Trust (WT88378).

#### **REFERENCES**


Chubb, C., Sperling, G., and Solomon, J. A. (1989). Texture interactions determine perceived contrast. *Proc. Natl. Acad. Sci. U S A* 86, 9631–9635.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 September 2013; accepted: 05 February 2014; published online: 20 February 2014*.

*Citation: Terhune DB, Song SM, Duta MD and Cohen Kadosh R (2014) Probing the neurochemical basis of synaesthesia using psychophysics. Front. Hum. Neurosci. 8:89. doi: 10.3389/fnhum.2014.00089*

*This article was submitted to the journal Frontiers in Human Neuroscience*.

*Copyright© 2014 Terhune, Song, Duta and Cohen Kadosh. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

## N1 enhancement in synesthesia during visual and audio–visual perception in semantic cross-modal conflict situations: an ERP study

*Christopher Sinke1,2 , Janina Neufeld1,3 , DanielWiswede4 , Hinderk M. Emrich1, Stefan Bleich1, Thomas F. Münte4 and Gregor R. Szycik1\**

*<sup>1</sup> Department of Psychiatry, Social Psychiatry and Psychotherapy, Hannover Medical School, Hanover, Germany*

*<sup>2</sup> Department of Neurology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany*

*<sup>3</sup> School of Psychology and Clinical Language Sciences, University of Reading, Reading, UK*

*<sup>4</sup> Department of Neurology, University of Lübeck, Lübeck, Germany*

#### *Edited by:*

*Nicolas Rothen, University of Sussex, UK*

#### *Reviewed by:*

*Nina Bien, Maastricht University, Netherlands Alodie Rey-Mermet, University of Bern, Switzerland*

#### *\*Correspondence:*

*Gregor R. Szycik, Department of Psychiatry, Social Psychiatry and Psychotherapy, Hannover Medical School, Carl-Neuberg-Straße 1, 30625 Hanover, Germany e-mail: szycik.gregor@mh-hannover.de*

Synesthesia entails a special kind of sensory perception, where stimulation in one sensory modality leads to an internally generated perceptual experience of another, not stimulated sensory modality.This phenomenon can be viewed as an abnormal multisensory integration process as here the synesthetic percept is aberrantly fused with the stimulated modality. Indeed, recent synesthesia research has focused on multimodal processing even outside of the specific synesthesia-inducing context and has revealed changed multimodal integration, thus suggesting perceptual alterations at a global level. Here, we focused on audio– visual processing in synesthesia using a semantic classification task in combination with visually or auditory–visually presented animated and in animated objects in an audio– visual congruent and incongruent manner. Fourteen subjects with auditory-visual and/or grapheme-color synesthesia and 14 control subjects participated in the experiment. During presentation of the stimuli, event-related potentials were recorded from 32 electrodes. The analysis of reaction times and error rates revealed no group differences with best performance for audio-visually congruent stimulation indicating the well-known multimodal facilitation effect. We found enhanced amplitude of the N1 component over occipital electrode sites for synesthetes compared to controls.The differences occurred irrespective of the experimental condition and therefore suggest a global influence on early sensory processing in synesthetes.

#### **Keywords: synesthesia, multimodal, EEG, N1, integration**

### **INTRODUCTION**

Synesthesia describes a specific kind of perception in which a particular stimulus in one sensory modality ("inducer") induces a concurrent perception in another sensory modality. Each kind of synesthesia can be defined by the specific inducer-concurrent pairing. The main characteristics of synesthesia are its consistency (Baron-Cohen et al., 1987; Simner and Logie, 2007) and automaticity (Mills et al., 1999): one inducer always triggers the same concurrent sensation, which cannot be suppressed or altered voluntarily. Grapheme-color synesthesia (GCS), in which achromatic letters, words or numbers are perceived in specific colors, has been extensively investigated and is believed to be one of the most common types (Simner et al., 2006). In auditory-visual synesthesia, sounds (e.g., music or single tones) can induce additional visual experiences, such as colors, forms, and textures (Ward et al., 2006; Neufeld et al., 2012a). Usually synesthetes have multiple types of synesthesia, suggesting a more global perceptual alteration underlying synesthesia rather than a specific one that only affects specific stimuli in two sensory modalities. Recent research suggests synesthesia to be an extreme form of multisensory processing within a continuous spectrum of normal perceptual processes involving multiple senses (Bien et al., 2012). Following this point of

view it is not surprising that synesthetes also show differences in multisensory processing not only restricted to the inducerconcurrent sensory modalities (Brang et al., 2012; Neufeld et al., 2012c; Sinke et al., 2012b) and that these differences are similar for both grapheme-color and audio–visual synesthetes (Neufeld et al.,2012c; Sinke et al., 2012b) indicating common sensory effects for different synesthesia phenotypes. Synesthesia is therefore not only characterized by specific synesthetic perception but rather these perceptions may be a tip of the iceberg indicating more global changes in sensory processing that are not necessary related to a specific inducer-concurrent coupling. However, up to now research on multimodal processing in synesthesia beyond typical inducer-concurrent perception is scarce.

To our knowledge, so far only three studies analyzed multimodal integration processes in synesthetes beyond the synesthetic perception (Brang et al., 2012; Neufeld et al., 2012c; Sinke et al., 2012b). Two of them focused on so-called double-flash illusion as described by Shams et al. (2000), in which a short flash is presented together with two short beep sounds while subjects have to state the number of perceived flashes. Subjects tend to report the occurrence of two flashes even though only one was presented. Regarding this effect the two mentioned studies found opposite effects: whereas Brang et al. (2012) reported an increased number of illusions in seven grapheme-color synesthetes, Neufeld et al. (2012c) found a decrease in 18 synesthetic subjects with GCS and/or auditory-visual synesthesia. The third study dedicated to this issue focused on two different multimodal effects (Sinke et al., 2012b). First, a reduced susceptibility to the so-called McGurk illusion (McGurk and MacDonald, 1976) was described in 19 synesthetes. In this illusion subjects watch a video and have to report what the person in the video says. Here the presentation of differing visual and acoustical information (video of a speaker saying "BA" dubbed with the audio track of the speaker saying "GA") leads to the fused perception of something new (usually "DA"). In the second part of the study, audio–visual speech enhancement was found to be reduced in the synesthesia group. Previously it was shown that in a noisy environment typical subject tends to additionally rely on the visual information during speech perception (i.e., mouth movement), depending on the quality of the acoustical signal (Ross et al., 2007). Here synesthesia subjects benefit less than control subjects from viewing articulatory lip movements in acoustically compromised situations. Therefore the study of Sinke et al. (2012b) shows for the first time, that subjects affected by synesthesia have deficits related to multimodal sensory processing that are important in our everyday life, namely in the speech perception. Thus these behavioral studies suggest global differences in multimodal sensory processing in synesthesia and further – resulting from these differences specific deficits related to basic natural sensory functions like speech perception.

Common models related to synesthesia focus primarily on typical inducer-concurrent couplings. Therefore they are based on data collected within a group of synesthesia subjects characterized by one specific inducer-concurrent coupling. Thus most data stems from the most available synesthesia group – GCS – though it is rarely reported if the investigated grapheme-color synesthetes experience also additional synesthesia types. Within these models proximal and distal causes of synesthesia have been distinguished (Ward, 2013). Differences in brain connectivity have been identified as a proximal cause. For example, the well-known cross-activation model of GCS suggests unusual direct connections between anatomically adjacent brain areas responsible for processing of inducer and concurrent (Ramachandran and Hubbard, 2001). As an alternative to this feed-forward mechanism with direct connections between unimodal sensory regions, indirect mechanisms based on feedback activity have also been discussed. According to this model – the disinhibited-feedback theory – synesthesia may be caused by disinhibited feedback from higher sensory or multimodal convergence brain sites (Grossenbacher and Lovelace, 2001). One good candidate for a synesthesia-related convergence site is the intraparietal cortex (IPC). This region receives mainly multimodal input (Bremmer et al., 2001) and shows structural differences in synesthetes (Weiss and Fink, 2009). It also shows activation differences in grapheme-color (Weiss et al., 2005; Sinke et al., 2012a) and in auditory-visual synesthetes (Neufeld et al., 2012a). Furthermore inhibitory transcranial magnetic stimulation (TMS) of the IPC disrupts the synesthetic Stroop effect (Esterman et al., 2006; Muggleton et al., 2007; Rothen et al., 2010), which is usually observed in grapheme-color synesthetes (Mattingley et al., 2001; Elias et al., 2003). Recent research shows also increased functional connectivity between this area and the primary auditory and visual cortices in audio–visual synesthetes (Neufeld et al., 2012b) and with the primary visual cortex in grapheme-color synesthetes (Sinke et al., 2012a). Thus it can be expected that synesthesia and synesthesia-related deficits in multisensory integration are related to aberrations within sensory-specific and higher sensory convergence brain sites and the communication among those.

As a possible distal cause of these connectivity differences and hence synesthesia, a deficit in pruning of synaptic connections has been hypothesized (Ward, 2013). Thus, synesthesia in adults could be a result of an altered development of the whole sensory system by deficient synaptic elimination. A very recent hypothesis based on findings of the graph theoretical network analysis states that synesthetes have a generally hyper-connected brain (Hanggi et al., 2011; Jancke and Langer, 2011) which may lead to alterations in multimodal integration processes at a global level within this population (Esterman et al., 2006; Mulvenna and Walsh, 2006).

Thus recent research gives first evidence for synesthesia as a global problem of multisensory processing with perception deficits that are affecting the speech processing and which may be related to both global alterations in brain connectivity and specific changes in communication between multimodal convergence brain sites and sensory-specific areas. Very little is known about the range of the multimodal speech-related deficits in synesthesia and the underlying mechanisms. Therefore, synesthesia research focusing on speech and speech-related perception as a multimodal phenomenon is needed.

Since synesthesia subjects show performance deficits in audio– visual speech perception under noisy environment, probably related to deficits in the integration of auditory stream with matching visual information served by vocalisatory lip movements, we decided to analyze audio–visual integratory processes at the semantic level. Under the assumption that synesthesia is related to global differences in brain connectivity leading to global changes in sensory perception and resulting from development problems of the central nervous system (pruning deficit), it should be possible to find differences in performance and in brain activation at diverse processing levels during cross-modal tasks. Such differences should be independent of the type of synesthesia, as defined by the specific inducer-concurrent coupling, and should have no relation to the synesthetic perception as such. To capture these effects we decided to utilize a simple multimodal perception task using different multi- and unimodal stimuli. It is known that object detection is faster for semantically congruent multimodal stimuli compared to unimodal stimuli (multimodal facilitation effect), whereas crossmodal conflict impairs the performance (Chen and Spence, 2010). Therefore we use a categorization task including three kinds of stimuli: semantically congruent and incongruent combinations of line drawings and sounds of animated and in animated objects and visual only presentations of line drawings. To capture the brain activation related to these processes we decided to use electroencephalography (EEG). This method allows analyzing brain activity with excellent temporal resolution and therefore is predisposed to distinguish early and late effects in sensory processing in an experimental setup. For both behavioral and EEG data, we expected differences between synesthesia and control subjects with reduction of audio–visual facilitation in synesthesia within the bimodal conditions and no differences in performance pattern in the unimodal visual condition.

## **MATERIALS AND METHODS SUBJECTS**

All study subjects gave written informed consent and the study was approved by the ethics committee of the Hannover Medical School. The subjects participated voluntarily and received a small monetary recompensation for their participation.

Control subjects (*n* = 14) and synesthesia subjects (*n* = 14) were matched for age (synesthetes: 36 ± 15 years, range 19–57, controls: 36 ± 14 years, range 22–61), gender (nine women per group), and general intelligence (IQ values for synesthetes: 119 ± 13 and controls: 112 ± 17) as assessed by the MWT-B – "Mehrfach–Wortschatz Test" (Lehrl et al., 1995). Data of two synesthesia subjects had to be excluded subsequently from the analysis due to strong artifacts. All subjects were native speakers of German with normal or corrected to normal vision and reported no history of neurological or psychiatric diseases or medication.

Synesthesia was assessed during an extensive interview. After the interview, five subjects were assigned to the audio–visual synesthesia (AVS) group, six to the GCS group, and three showed both kinds of synesthesia (**Table 1** contains information regarding additional synesthetic inducer-concurrent pairings within our synesthesia population). All subjects underwent additional testing with an offline MATLAB version of the synesthesia battery (http://www.synesthete.org/). In the battery, numbers from 0 to 9 (10) and the letters from the alphabet from A to Z (26) are presented and grapheme-color synesthetes have to select a color which matches their synesthetic experience best, while controls have to choose a color which they think fits best to the item. Additionally, we modified the battery for subjects with AVS using 36 tones similar as used by Ward et al. (2006). In this test, synesthetes are asked to choose the color which matches their experienced synesthetic color induced by the tone best, non-synesthetes are asked to select the color which they think to fit best to the tone. Subjects with both kinds of synesthesia participated in both versions of the battery. Each item of the synesthesia battery was presented three times in randomized order. To assess consistency the geometric distance in RGB color space between the three runs of each sound was calculated for each subject (Eagleman et al., 2007). More consistent color choices lead to a lower consistency score, as more consistent color choices for each sound result in more similar RGB values and thus a smaller difference between the RGB values. For grapheme-color synesthetes a threshold value of 1 was chosen as suggested by Eagleman et al. (2007). All grapheme-color synesthetes showed consistency scores lower than 1 (synesthesia group 0.59 ± 0.18, control group 2.09 ± 0.68). Since a similar threshold has not been defined for auditory-visual synesthesia, we merely show that the group of auditory-visual synesthetes was more consistent than the control group, as suggested by Ward et al. (2006). The group of audio–visual synesthetes showed significantly lower consistency scores (1.15 ± 0.45) than the control group (2.03 ± 0.47).

#### **STIMULI AND TASK**

For visual stimulation we used line drawings of animals and inanimate objects selected from the Snodgrass and Vanderwart (1980) database presented centrally on a black screen (approximately 9◦ visual angle horizontally and 6◦ vertically). Complex natural sounds from the MULTIMOST stimulus set served as auditory stimuli (Schneider et al., 2008). Three conditions were used (**Figure 1**): in the audio–visual congruent condition the line drawing matched the presented sound (e.g., drawing of lion accompanied by a lion's roar). This condition should lead to crossmodal enhancement. In the audio–visual incongruent condition a semantic mismatch between both modalities (e.g., a drawing of a lion accompanied by a telephone ring) was introduced. This condition is expected to lead to cross-modal conflict between visual and auditory information. The unimodal control condition comprised only visual stimulation with line drawings without sound presentation. Each stimulus was presented for duration of 400 ms. Auditory and visual stimuli in bimodal conditions were


*Asterisk indicates subjects excluded from the analysis.*

presented concurrently without time delay between onsets. The inter-stimulus interval, during which fixations cross was presented in the center of the screen, varied between 2 and 3 s in 100 ms steps. For each experimental condition (congruent, incongruent, and visual only) 80 stimuli were presented (20 different animate and 20 different inanimate stimuli, all stimuli were presented twice to increase number of events). Stimuli of all experimental categories were presented in random order. The experiment lasted about 10 min and comprised 240 stimuli in total. Participants were required to categorize each visual stimulus as either animate or inanimate as fast as possible by pressing the left or right mouse button. Before the experiment each participant completed a practice run of 10 trials with visual and acoustical congruent stimuli not presented in the main experiment to ensure that the participant understood the task. All stimuli were presented on a 19 flat screen with a resolution of 1280 × 1024 pixels. Sounds were adjusted individually to a comfortable listening level and presented on standard loudspeakers in binaural mono. The experiment was implemented using Presentation software (Neurobehavioral Systems, Inc., Albany, CA, USA).

#### **DATA ACQUISITION AND PRE-PROCESSING**

After application of the electrodes, participants were seated in a separate EEG recording chamber with dimmed light. Participants rested their hands on a computer mouse placed on the table infront of them, responding with their left and right index fingers. Electroencephalographic (EEG) activity was recorded continuously using an Active Two head cap and the Active Two BioSemi system (BioSemi, Amsterdam, Netherlands). Signals were recorded from 32 positions including all standard locations of the 10/20 system using active electrodes in an elastic cap. Recording of additional electrodes to record eye artifacts was not necessary, since the analysis software provides estimation of eye artifacts from frontocentral scalp electrodes (FP1, FP2). As usual for BIOSEMI, two additional electrodes (common mode sense, CMS, and driven right leg, DRL) were used as reference and ground electrodes during

recording. Bioelectric signals were amplified with a sampling rate of 1024 Hz and stored using ActiView software (BioSemi) with decimation/anti-aliasing filter (5th order sinc filter, low-pass with −3 dB at 0.2035 Hz \* 1024 Hz) applied to the data streamed to file. Prior to ERP analysis, EEG data were downsampled to 256 Hz and re-referenced to common average reference. We decided to use this reference method instead of re-referencing the signal to specific electrodes (e.g., averaged mastoids) because we were also interested in potential stimulation effects over the auditory cortex. A high-pass filter (1 Hz to remove low frequency drifts) and a notch filter (peak at the line frequency of 50 Hz) were applied. In the recent literature the use of high pass filtering is discussed critically. Some authors suggests to analyze data without filtering (Vanrullen, 2011) while others suggest maximal filter cut off frequency of 0.1 Hz (Acunzo et al., 2012), or values higher than 0.1 Hz (Widmann and Schroger, 2012) or lower than 1 Hz (Rousselet, 2012). Thus the use of filters affects the EEG signal in the time domain resulting in reduced precision and artifacts. We decided to use a relative high cut off frequency of 1 Hz for the high pass filter in our analysis. This relatively high value can result in some serious artifacts as shown by Acunzo et al. (2012) where the filtered signal shows artificial differences between conditions within the same experimental group. In our recent study we primary focus on differences between groups. Therefore filter settings should have the same impact on EEG signal in all experimental groups and leave the potential group difference unaffected. Indeed in studies dedicated to analysis of the impact of high pass filter setting on group differences no effects for early EEG components were found (Ebmeier et al., 1992; Goodin et al., 1992). We therefore believe that the chosen filter settings should not influence our analysis focused on group effects. Ocular contributions to the EEG were corrected using blind component separation, SOBI (Joyce et al.,2004), which has been shown to be superior to other artifact correction procedures (Kierkels et al., 2006). Rejection of non-EOG-artifacts was accomplished using individualized peak-to-peak-amplitude criteria based on visual and semi-automatic inspection implemented in BESA software (www.BESA.de). To remove high frequency noise, ERPs were 30 Hz low-pass filtered prior to statistical analysis and graphical display. Grand-average ERPs were generated separately for both groups. ERPs were time-locked to the onset of the stimulation.

#### **DATA ANALYSIS**

Behavior was assessed by reaction time (RT) and error rate (ER). The data was analyzed by means of 2 × 3 ANOVA with main between-subjectsfactor synesthesia (synesthesia vs. control group) and within-subject factor stimulation (audio–visual congruent vs. audio–visual incongruent vs. only visual stimulation).

Electroencephalography data was analyzed in two steps. The first step contained exploratory inspection of all electrodes for possible differences and relevant time windows. In the second step early ERP effects were quantified by analysis of the greatest negative peak amplitude within time window from 80 to 180 ms and late ERP effects were quantified by a mean amplitude measure between 200 and 400 ms. Since visual N1 component consist of a complex of at least three separate subcomponents that are associated with current flows over frontal (peaking at 140 ms), parietal (150 ms), and

occipital (170 ms) scalp areas (Luck, 1995) analysis of this component should involve this time range of EEG signal. We decided therefore to use a time window from 80 till 180 ms used already by others for analysis (Johannes et al., 1995; Vogel and Luck, 2000) of this component. The time window for analysis of the late component was chosen to grasp possible effects on the N400 component usually modulated by semantic mismatch of the incoming information (Kutas and Federmeier, 2011). EEG data was analyzed first for global group effects and effects of stimulation with localization effects for ventro-dorsal and left-right axis. For this purpose a 2 × 3 × 2 × 4 ANOVA model was designed. This model contained one main between-subjects factor synesthesia (synesthesia vs. control group), one within-subject factor stimulation (audio–visual congruent vs. audio–visual incongruent vs. only visual stimulation) and further within-subject factors laterality (left vs. right) and electrodes (frontal vs. central vs. parietal vs. occipital) using electrodes along fronto-caudal and left-right axis (F3, C3, P3, O1, F4, C4, P4, and O2). Degrees of freedom are provided uncorrected; whenever necessary, *p*-values are Greenhouse–Geissercorrected to account for possible violations of the sphericity assumptions.

Since both audio–visual conditions simultaneously included visual/auditory stimulation, both of them should show a mixed ERP based on visual and auditory potentials. In contrast, the visual stimulation control condition included visual potentials only, which enables us to see group differences when confronted with visual stimulus material only. Therefore, the visual condition is presented independently of the audio–visual conditions in the **Figure 2**.

#### **RESULTS**

#### **BEHAVIORAL RESULTS**

Behavioral results are summarized in **Table 2**. The ANOVA synesthesia × stimulation on ER data revealed no significant effects (stimulation *F*2,48 < 1, group *F*1,24 = 2.2 *p* = 0.15, interaction *F*2,48 < 1). The ANOVA synesthesia × stimulation on response time (RT) data revealed significant effect of stimulation (*F*2,48 =3.1 *p*=0.05). Both the groupfactor synesthesia (*F*1,24 <1) and the interaction of synesthesia with stimulation (*F*2,48 < 1) was not significant. *Post hoc* tests revealed faster responses for audio– visual congruent than incongruent stimuli (*t*<sup>25</sup> = 2.7 *p* =0.01) and for visual only than audio–visual incongruent (*t*<sup>25</sup> = 2.1 *p* = 0.05). The difference between visual only and audio–visual congruent stimuli was not significant (*t*<sup>25</sup> = 0.4 *p* = 0.72).

#### **ERP RESULTS**

In the overall ANOVA comparing factors synesthesia, stimulation, laterality, and electrodes for the peak analysis (N1 component) we found a significant effect of synesthesia (*F*1,24 = 5.4, *p* = 0.03) and a significant effect of electrodes (*F*3,72 = 32.4, *p* = 0.00) comparing signal from frontal (F3 and F4) vs. central (C3 and C4) vs. parietal (P3 and P4) vs. occipital (O1 and O2) electrodes. We also found a significant interaction between synesthesia and electrodes (*F*3,72 = 3.8, *p* = 0.04). Thus in the next step we analyzed by means of ANOVA the effects of group and stimulation in the frontal (F3, F4), central (C3, C4), parietal (P3, P4), and occipital (O1, O2) electrodes separately. We

found no significant effects for the frontal, central, and parietal electrodes.

For the occipital electrodes we found a significant stimulation effect (*F*2,48 = 7.0, *p* = 0.08) as well as group effect (*F*1,24 = 5.2, *p*=0.03) but no significant interaction between both. *Post hoc* tests revealed no significant differences in processing of audio–visual congruency (*t*-test on mean data of congruent vs. incongruent was n.s.), but congruent audio–visual vs. visual only stimulation (*t*<sup>25</sup> = 2.9, *p* = 0.08, −8.4 ± 6.0 μV vs. −9.7 ± 5.8 μV) and incongruent audio–visual vs. visual only stimulation (*t*<sup>25</sup> =2.2, *p*=0.04, −8.7 ± 5.9 μV vs. −9.7 ± 5.8 μV). Thus the described stimulation effect rises from the processing difference of multimodal vs. unimodal stimulation as such, irrespective whether multimodal stimulation was congruent or incongruent. The group effect above the occipital electrodes was due to a global difference in processing of both visual only and audio–visual stimuli with stronger negativity in the synesthesia group. We decided to calculate also *post hoc* tests for group differences although there was no significant interaction effect of both main factors. The reason for this was the relatively small synesthesia subject population analyzed. Therefore *post hoc t*-tests revealed a trend for significance for comparison between control and synesthesia group in audio–visual congruent condition (*t*<sup>24</sup> = 1.9, *p* = 0.06, −6.3 ± 3.6 μV vs. −10.7 ± 7.5 μV) and virtually significant difference for audio-visual incongruent condition (*t*<sup>24</sup> = 2.1, *p* = 0.05, −6.6 ± 3.6 vs. −11.1 ± 7.1 μV) and significant difference for visual only condition (*t*<sup>24</sup> = 2.8, *p* = 0.01, −7.1 ± 3.1 vs. −12.7 ± 6.7 μV). Re-analysis using mean amplitude in a time-window from 80 to 180 ms replicated those findings.

No synesthesia (*F*1,24 < 1) or stimulation (*F*2,48 < 1) effects were found when analyzing the late components between 200 and 400 ms. The factor laterality was also not significant (*F*1,24 = 2.2, *p* = 0.15) but the factor electrodes showed similar to N1 component significant effect (*F*3,72 = 4.9, *p* = 0.03). *Post hoc t*-test revealed differences between the central and parietal electrodes (*t*<sup>25</sup> = 5.4, *p* = 0.00, −0.1 ± 1.7 vs. 2.6 ± 2.0 μV) and between frontal and parietal electrodes (*t*<sup>25</sup> = 3.7, *p* = 0.00, −0.5 ± 2.4 vs. 2.6 ± 2.0 μV).

## **DISCUSSION**

The aim of the study was twofold. First we were interested in global multisensory perception alterations in synesthesia independent of the specific inducer-concurrent couplings. Second we focused on neuronal activation underlying these perceptual processes. Thus, audio–visual semantic matching was used to capture multisensory processing in synesthesia at the global level. We hypothesized, based on the idea of general hyperconnectivity in synesthesia (Hanggi et al., 2011), that we would find differences in multisensory integration processes unrelated to synesthetic sensations between synesthetes and controls, i.e., in an audio–visual semantic categorization task. Particularly with regard to previous behavioral data that suggested the reduction of audio–visual integration in synesthetes as indicated by a reduced number of audio–visual illusions (Neufeld et al., 2012c; Sinke et al., 2012b) and the reduction of multisensory facilitation in speech perception (Sinke et al., 2012b), we expected global effects related to the behavior and the EEG signal.

On the behavioral level, we didn't find any effects between groups in the experiment. Compared to controls, synesthesia subjects showed similar ERs and response speeds in audio–visual congruent and incongruent experimental events. Also, unimodal visual processing was similar in both groups. There are different possible explanations for the lack of behavioral group-effects in this study. The first one is related to the stimulation used in our experiments, i.e., it could be possible that stimulation was not able to evoke the expected effects. However, we observed in our data the well known multisensory facilitation effect (Chen and Spence, 2010) with faster responses for audio–visual congruent stimuli. Therefore our stimulation was evidently sufficient for audio–visual semantic integration. Interestingly the multisensory facilitation effect was not accompanied by significant EEG signal

#### **Table 2 | Behavioral results.**


differences between audio–visual congruent and incongruent events. The second possibility is that previously described deficits in multisensory integration of synesthesia subjects (Neufeld et al., 2012c; Sinke et al., 2012b) are related to early processing stages involving a more basal stimulus analysis than the semantic integration/matching, which relies on the conceptual knowledge. Thus it is possible that the used stimuli in form of line drawings and complex sounds involve other integratory mechanisms than those related to temporal correspondence and spatial congruence of stimuli. Lastly, the lack of behavioral effects may be resulting from compensatory mechanisms on the neuronal level within the synesthesia group. Subjects with synesthesia might have to spend more attention to manage their interaction with the environment despite the often-reported confusion caused by synesthetic sensations. Thus they may develop strategies during their life to manage sensory input from different sensory channels separately. A good candidate for such compensatory strategy could be the control over attentional processes related to global sensory perception in synesthesia, which allows better separation of sensory information coming from different modalities. As a side effect of such strategy, or in other words, as price for successful interaction with the environment, synesthesia-specific reduction in multimodal integration could arise. Some evidence for this idea is provided by the finding that synesthesia subjects show a negative relation between their susceptibility for audio– visual illusions and their age (Neufeld et al., 2012c). This finding suggests that subjects with synesthesia are reducing their tendency to integrate multisensory information throughout their life.

Consistent with the idea of additional neuronal mechanisms related to multimodal sensory processing in synesthesia and our experimental hypothesis, we observed differences in the EEG signal of synesthetes and controls. Therefore, the main finding of this study is the global difference in the N1 negativity over occipital electrodes between synesthesia subjects and controls. This difference was unrelated to the experimental condition as synesthesia subjects showed a much stronger negativity for audio–visual congruent, incongruent as well as for unimodal visual-only stimuli. This not only indicates differences in multimodal but also in unimodal processing of synesthetic non-inducing stimuli in synesthetes and may reflect global alterations in sensory processing. Additionally we found no specific ERP effects of audio–visual congruency, neither within the synesthesia nor within the control group. Such effects were expected, since semantically congruent

multimodal stimuli facilitate object detection and recognition whereas incongruent stimuli induce multimodal conflict and impair performance (Chen and Spence, 2010) and since we found this effect in the behavioral data in this study. Semantic mismatches are associated with a modulation of the so-called N400 component, which has been demonstrated for language (Kutas and Hillyard, 1980) and pictorial (Ganis and Kutas, 2003) material. The lack of audio–visual congruency/incongruency effects in our study might be explained by visual dominance effects in audio–visual conflict situations with ambiguous auditory and visual input (Yuval-Greenberg and Deouell, 2009). Another explanation for the lack of audio–visual congruency effects in the EEG signal in spite of the existence of behavioral multisensory facilitation effect in our data could be based on the small difference between the RTs for congruent vs. incongruent stimuli (about 7 ms). We consider it as possible that our experimental setting was not sufficient to capture brain correlates of such small behavioral effects. To summarize, global N1 group difference might rather be related to a generally altered visual processing in synesthetes and not to altered multisensory integration processes.

Alterations of early visual processing as measured by EEG in synesthetes have already been shown previously (Barnett et al., 2008). Barnett et al. (2008) used simple stimuli that do not elicit synesthetic color experiences and which are either mainly processed via the parvocellular or the magnocellular visual pathway. Sensory-perceptual differences in synesthetes relative to nonsynesthetes in response to both types of stimuli were observed with enhanced processing of parvocellular stimuli (high contrast, high spatial frequency stimuli including color stimuli) reflected by an enhanced C1 component at 65–85 ms and a trend of decreased response to magnocellular stimuli (low contrast, low spatial frequency gray scale stimuli). The authors argue that these differences in early evoked visual potentials are a marker of widespread connectivity differences, which might be the cause of both, synesthesia as well as unrelated sensory processing differences. Alternatively, differences in early sensory processing (enhanced parvocellular and reduced magnocellular processing) might determine synesthesia by indirectly increasing a tendency to develop the paired associations of inducing stimuli with color percepts (as both, graphemes as well as colors, are more parvocellularly processed). In our study the visual stimuli were white line drawings on black background, which were therefore high in contrast and rather high in spatial frequency and therefore parvocellular in nature, which fits to the findings of Barnett et al. (2008). In line with the finding of enhanced ERPs evoked by stimuli mainly recruiting the parvocellular pathway, enhanced unimodal perception in the modality of the concurrent (which was vision or tactile sensation) has been reported in synesthetes (Banissy et al., 2009) which provides further evidence for the idea of a generally altered early unimodal processing and fits to the enhanced early N1 component over visual cortex found in our study.

Alterations of the N1 over occipitotemporal sites have been shown in grapheme-color synesthetes when exposed to numbers inducing synesthetic colors, which could either be congruent or incongruent to the meaning of a preceding sentence (Brang et al., 2008; Brang et al., 2011). More precisely, between 100 and 150 ms after onset of the sentence final stimulus (which was the number), ERPs to numbers inducing contextual congruent concurrent were more negative than ERPs to incongruent numbers in the synesthete group only. In contrast we found increased N1 component for different kinds of stimuli. Although both studies focused on different perception aspects (expectancy of content within consecutive presented sentence ending with synesthetic inducer vs. concurrent audio–visual stimulation), effects related to N1 component were found. The authors of the above mentioned study argue that their finding might indicate differences in attention shift processes dependent on the congruency of the inducers occurring on a rather early sensory level and that the observed N1 effects may reflect enhanced visual processing of contextually appropriate graphemes in the sense that fulfilled semantic expectations facilitate the grapheme discrimination as well as its synesthetic color. We go a step forward and argue that the finding of an enhanced N1 component in the current study might also be the result of an attention-related, facilitated sensory processing, but in contrast to the study by Brang et al. (2008) it occurred within the synesthesia group on a basic visual level unaffected by context (congruent or incongruent additional auditory stimuli) and without inducing synesthetic concurrents.

Influence of attention on the visual N1 in non-synesthetic individuals has already been reported in the context of both multisensory integration (Talsma andWoldorff,2005) and unisensory visual processing (Vanvoorhis and Hillyard, 1977; Harter et al., 1982; Clark and Hillyard, 1996). Following the idea that attention already has particular impact on early stages of sensory processing in synesthesia, involvement of attention-related parietal cortex in perception could be expected. In fact, evidence for the parietal cortices key role in synesthetic perception comes from several neuroimaging studies with groups of grapheme-color synesthetes (Rouw and Scholte, 2010; Van Leeuwen et al., 2010; Sinke et al., 2012a), spatial sequence synesthetes (Tang et al., 2008) and auditory-visual synesthetes (Neufeld et al., 2012a). Importantly, connectivity analyses identified an area in the parietal cortex which showed stronger connections with primary sensory areas in synesthetes (Neufeld et al., 2012b; Sinke et al., 2012a). This is consistent with a model of parietal modulation of sensory processing which has been found to explain neuroimaging data of associator synesthetes (Van Leeuwen et al., 2011).

To summarize, in the behavioral data we found no group differences between synesthesia subjects and control subjects regarding the ER and the response speed but rather a group independent multisensory facilitation effect with faster responses for audio– visual congruent stimuli. The lack of behavioral group differences was contrasted by a global group difference in N1 for the occipital electrodes. Here, synesthesia subjects showed a stronger negativity for different kinds of stimuli. Taken together, our results give evidence for global early sensory processing alterations in synesthetes concerning a very basic level of visual processing. These early visual processing differences might either be the result of an altered connectivity within the visual cortex or of a modulation of visual processing mediated by (parietal) influences related to attention.

#### **ACKNOWLEDGMENTS**

We acknowledge support by Deutsche Forschungsgemeinschaft: "Open Access Publication." This work was supported by the German Research Foundation, SFB936/A4 (Christopher Sinke).

### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 July 2013; accepted: 10 January 2014; published online: 30 January 2014. Citation: Sinke C, Neufeld J, Wiswede D, Emrich HM, Bleich S, Münte TF and Szycik GR (2014) N1 enhancement in synesthesia during visual and audio–visual perception in semantic cross-modal conflict situations: an ERP study. Front. Hum. Neurosci. 8:21. doi: 10.3389/fnhum.2014.00021*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2014 Sinke, Neufeld, Wiswede, Emrich, Bleich, Münte and Szycik. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Impaired acquisition of novel grapheme-color correspondences in synesthesia

## *David Brang1,2\*, Michael Ghiam2 and Vilayanur S. Ramachandran2*

*<sup>1</sup> Department of Psychology, Northwestern University, Evanston, IL, USA*

*<sup>2</sup> Department of Psychology, University California San Diego, La Jolla, CA, USA*

#### *Edited by:*

*Nicolas Rothen, University of Sussex, UK*

#### *Reviewed by:*

*Gregor Thut, University of Glasgow, UK Danko Nikolic, Max Planck Institute*

#### *for Brain Research, Germany \*Correspondence:*

*David Brang, Department of Psychology, Northwestern University, 2029 Sheridan Road, Evanston, IL 60208-2710, USA e-mail: david.brang@ northwestern.edu*

Grapheme-color synesthesia is a neurological phenomenon in which letters and numbers (graphemes) consistently evoke particular colors (e.g., A may be experienced as red). These sensations are thought to arise through the cross-activation of grapheme processing regions in the fusiform gyrus and color area V4, supported by anatomical and functional imaging. However, the developmental onset of grapheme-color synesthesia remains elusive as research in this area has largely relied on self-report of these experiences in children. One possible account suggests that synesthesia is present at or near birth and initially binds basic shapes and forms to colors, which are later refined to grapheme-color associations through experience. Consistent with this view, studies show that similarly shaped letters and numbers tend to elicit similar colors in synesthesia and that some synesthetes consciously associate basic shapes with colors; research additionally suggests that synesthetic colors can emerge for newly learned characters with repeated presentation. This model further predicts that the initial shape-color correspondences in synesthesia may persist as implicit associations, driving the acquisition of colors for novel characters. To examine the presence of latent color associations for novel characters, synesthetes and controls were trained on pre-defined associations between colors and complex shapes, on the assumption that the prescribed shape-color correspondences would on average differ from implicit synesthetic associations. Results revealed synesthetes were less accurate than controls to learn novel shape-color associations, consistent with our suggestion that implicit form-color associations conflicted with the learned pairings.

#### **Keywords: synesthesia, synaesthesia, language, learning, grapheme**

## **INTRODUCTION**

Synesthesia is an involuntary experience in which stimulation of one sensory modality causes activation in a second, separate modality. In two of the most common forms, hearing sounds or viewing graphemes (letters and numbers) elicits the experience of a specific color (sound-color and grapheme-color synesthesias; Baron-Cohen et al., 1996; Cytowic and Eagleman, 2009). For example, to synesthete EA the letter C always appears a vivid yellow, irrespective of its actual color. These sensory percepts typically begin early in childhood and remain extremely consistent throughout one's lifetime. Further, synesthesia runs in families (Baron-Cohen et al., 1996; Ward and Simner, 2005; Asher et al., 2009; Brang and Ramachandran, 2011), suggesting it is a heritable trait. While research over the past decade has done well to establish the validity of synesthetes' subjective reports (e.g., Ramachandran and Hubbard, 2001a) as well as the perceptual reality of these experiences (Palmeri et al., 2002), the manner in which synesthetic associations develop from infancy to adulthood remains an active matter of debate.

The neural substrates underlying grapheme-color synesthesia have been thoroughly studied using both psychophysical tests and functional imaging (Nunn et al., 2002; Palmeri et al., 2002; Hubbard et al., 2005). Upon viewing achromatic numbers or letters, grapheme-color synesthetes show co-activation of grapheme regions in the posterior temporal lobe and color area V4, which has been suggested to give rise to the concurrent sensation of color (Hubbard et al., 2005; Sperling et al., 2006). This pattern of activation in synesthetes is consistent with research on crossmodal integration in the normal population revealing the engagement of early sensory regions in multimodal processes (Driver and Spence, 2000; Schroeder and Foxe, 2005; Kayser and Logothetis, 2007). Ramachandran and Hubbard (2001a) proposed that this cross-activation is driven by an excess of neural connections in synesthetes, possibly due to decreases in neural pruning between typically interconnected areas, with a recent update of this model introduced by Brang et al. (2010). In support of this theory, a number of studies have demonstrated anatomical differences in the inferior temporal lobes of synesthetes, near regions related to grapheme and color processing, including increased fractional anisotropy as assessed by diffusion tensor imaging (Rouw and Scholte, 2007), and increased gray matter volume, as assessed by voxel-based morphometry (Jäncke et al., 2009; Weiss and Fink, 2009).

While the developmental onset of synesthesia is a matter of active debate, it is well established that the specific synesthetic pairings (such as a yellow C for synesthete EA) evolve over time biased by statistical properties of the environment. One strong contributing factor is the native language of a synesthete, given that some letters tend to take on the colors related to prominent linguistic elements (e.g., R is typically red and G typically green based on the initial letter of the color-word in English; Simner et al., 2005) and as more frequent exposure to a particular grapheme enhances the perceived luminance of its associated synesthetic color (i.e., more frequent letters and numbers tend to be associated with more luminant synesthetic colors; Cohen Kadosh et al., 2007; Smilek et al., 2007; Beeli et al., 2008). Furthermore, some element of imprinting from early childhood may bias grapheme-color associations, such as those from colored refrigerator magnets (Witthoft and Winawer, 2013).

Given that synesthetic associations change throughout development, it has been proposed that prior to learning a character set for a particular language, synesthetic colors are initially associated with shapes and basic forms (Maurer, 1993; Brang et al., 2010; Wagner and Dobkins, 2011). Indeed, prior to learning graphemes, infants and toddlers associate Xs with the color black and Os with white (Spector and Maurer, 2008). These shape-color associations may permeate into adulthood as more similarly shaped letters evoke more similar colors in synesthesia (Brang et al., 2011a; Jürgens and Nikolic, 2012; Watson et al., 2012 ´ ) thought to occur from latent shape-color associations. In addition, some synesthetes report varying saturation of synesthetic colors by letters of different case and font suggesting an import of letter shape.

Contrary to this view, several studies highlight the influence of semantic information on synesthetic colors (e.g., Mroczko et al., 2009; Jürgens and Nikolic, 2012 ´ ) and instead propose that graphemes evoke ideas which in turn elicit synesthetic colors (i.e., "ideaesthesia"). In this model, low-level elements of a grapheme, such as shape, do not evoke synesthetic colors on their own. Instead synesthetic colors are evoked only when the synesthete extracts semantic meaning from the grapheme in isolation or in the context of a word or sentence (Jürgens and Nikolic, 2012 ´ ). Indeed, synesthetic colors are modulated by context, such that an ambiguous letter evokes the color corresponding to the perceived letter (e.g., Ramachandran and Hubbard, 2001b).

As synesthetic associations tend to transfer to novel learned scripts (e.g., Mroczko et al., 2009) and as synesthetes can learn new associations (Jürgens and Nikolic, 2012; Blair and ´ Berryhill, 2013) the question remains of whether latent color associations exist for the novel scripts based on their form (such that shapes in the novel character set that are similar to letters and numbers will take on similar colors). To test for the presence of latent color associations for novel characters, 15 synesthetes and 15 controls were trained to learn novel shape-color corresponances with the goal of causing conflict with latent associations, as the proscribed shape-color correspondences would on average differ from implicit synesthetic associations.

## **MATERIALS AND METHODS**

#### **PARTICIPANTS**

Data were collected from 15 synesthetes (mean age = 23.0; 13 females; 14 right-handed) and 15 non-synesthetic controls (mean age = 20.2; 10 females; 15 right-handed). All participants were fluent English speakers. Synesthetes were recruited via fliers posted on the UCSD campus, as well as similar ads on the web. Synesthesia was confirmed by means of consistency matching as well as reaction time testing for color congruency, standardized by Eagleman et al. (2007). All participants gave signed informed consent prior to the experiment, and participated either for cash or in fulfillment of course requirement. None of the control participants reported any known forms of synesthesia.

#### **DESIGN AND PROCEDURE**

Synesthetes first selected six novel graphemes from a set of 12 (**Figure 1**) that met the following requirements: (1) the shape elicited no synesthetic color, (2) the shape was unfamiliar to the subject, and (3) the shape did not evoke any strong conceptual association. After grapheme-selection, one color (red, orange, yellow, green, blue, purple) was randomly paired with each one of the six novel graphemes. Subjects were instructed to memorize six novel grapheme-color associations, one grapheme with each one of the six colors, but were not informed of the specific parings. Each control's stimulus set was matched to one synesthete's set of grapheme-color stimuli.

Subjects participated in two training tasks: a Guess-and-Check paradigm and a Color Congruency paradigm. In the Guess-and-Check task, subjects were presented with one of the six graphemes on each trial and instructed to speak aloud the associated color. Black target graphemes were presented centrally in Arial font, 6◦ vertical visual angle, against a gray background. Graphemes were present until response, preceded by a blank screen for 250 ms, and followed immediately after response by the same grapheme in the congruent color for 1000 ms. The first six trials reflected guesses as subjects had not been previously exposed to the associations and were excluded from analyses. Two hundred and thirty four trials (39 for each grapheme) followed these initial six trials. Accuracy was scored by an experimenter present for the duration of the study and reaction time was coded by EPrime Serial Response Box microphone.

Immediately following the first task, subjects participated in the Color Congruency paradigm. Subjects were presented with the same stimuli in either a congruent or incongruent color (50% of trials each). One incongruent color was yoked to each grapheme to balance color and stimulus probabilities. Graphemes were present on screen until response and subjects responded via button presses (EPrime Serial Response Box) to indicate whether a grapheme was congruently or incongruently colored. After response, a feedback screen (1500 ms) was provided to the subjects. Subjects progressed through 240 trials in total.

**FIGURE 1 | Grapheme set from which synesthetes selected six shapes that elicited no synesthetic color, were unfamiliar to the subject, and did not evoke any strong conceptual association.**

#### **ANALYSIS**

For the Guess-and-Check paradigm, synesthetes and controls were first compared according to the total number of errors using a paired *t*-test, matching synesthetes and controls based on learned associations. To examine whether groups differed in the length of time taken to learn shape-color associations or in the persistence of errors following reaching criterion, we conducted two additional tests. First, a learning curve and 95% confidence interval were computed for each individual subject using a state-space smoothing algorithm (Smith et al., 2004) to identify the trial on which subjects reliability performed better than chance for the remainder of the experiment (0.167 probability given the six possible colors; all default parameters were used in this analysis and the toolbox can be found at www.neurostat.mit.edu/behaviorallearning). The first trial on which this criterion was reached was compared across the groups using a paired *t*-test. In order to examine the persistence of errors after reaching criterion, a paired *t*-test was used to compare synesthete's and control's accuracy on trials following the trial on which each subject reached criterion. In the color congruency paradigm, synesthetes' and controls' mean accuracy and mean response times were each compared using paired *t*-tests.

#### **RESULTS**

The initial paradigm required subjects to learn the associated color for six novel graphemes through a guess-and-check task, with feedback. Synesthete and control groups average performance throughout the Guess-and-Check paradigm are displayed in **Figure 2**; also see **Figure S1**. In terms of the total number of mistakes made throughout this task, synesthetes exhibited a trend of a greater number of errors (mean 34.8) relative to controls (mean 24.9), *t(*14*)* = 1*.*77, *p* = 0*.*098. In order to examine whether this difference was due to synesthetes' difficulty in the acquisition or maintenance of the novel associations over time we conducted two additional analyses.

First, criterion-learning thresholds were estimated by calculating the probability of a correct response on any given trial, based on a state-space smoothing algorithm (Smith et al., 2004). This analysis identified the trial for which all subsequent responses exceeded chance performance (**Figure 3**), and comparison between the groups suggested no difference in learning rate between synesthetes (mean trials to criterion 24.7) and controls (mean trials 29.8), *t(*14*)* = 0*.*72, *p* = 0*.*486. In order to examine whether synesthetes and controls differed in their ability to maintain these associations throughout the task after reaching criterion, accuracy was compared for all trials following the criterion trial. Results identified significantly impaired performance in synesthetes (mean accuracy 91.7%) relative to controls (96.3%), *t(*14*)* = 3*.*59, *p* = 0*.*003, indicating a difficulty in maintaining these novel grapheme-color associations for synesthetes (**Figure 4**).

The second paradigm measured subjects' ability to identify congruently colored graphemes. Synesthetes displayed a non-significant trend of lower accuracy through this task (mean 95.6% correct) relative to controls (mean 97.8%), *t(*14*)* = 1*.*65, *p* = 0*.*122. No reliable difference in response time was observed between synesthetes (mean 834 ms) and controls (806 ms), *t(*14*)* = 0*.*42, *p* = 0*.*679.

## **DISCUSSION**

The present paradigm examined the ability of synesthetes and controls to learn novel grapheme-color correspondences. Findings across two paradigms with the same subjects indicate that synesthetes were impaired in their ability to maintain novel, enforced grapheme-color associations. These results suggest that synesthetes are not simply better at maintaining arbitrary associations *per se*, as has been suggested previously (e.g., Ramachandran and Hubbard, 2001b), but instead argues for the presence of latent grapheme-color associations for novel character sets.

**FIGURE 3 | Learning curve for a representative subject.** Solid line (dark green) reflects the probability of a correct response on any given trial. The light green cloud reflects the 95% confidence interval for this probability. Learning criterion was reached when a subject's accuracy exceeded chance (dotted gray line) with 95% certainty (marked with the blue arrow at trial number 28). The plot begins at trial number seven as the first six trials were guesses and were excluded from all analyses, and only shows results for trials up to 100 for illustrative purposes.

Described as the neonatal synesthesia hypotheses, one theory suggests that synesthesia is present in all individuals at or near birth and fades over time due to excessive neural pruning and re-weighting of synaptic connections (Maurer, 1993). However, as letters and number acquisition occurs relatively late in development (on average at 3 years of age; e.g., Wynn, 1990), if synesthetic associations are present early in life they most likely involve shape-color associations. In line with this, young children show consistent associations between shapes and colors (Spector and Maurer, 2008; Wagner and Dobkins, 2011) and synesthetic colors remain fluid (some letters-color associations show large variability over time) at least until age 10 (Simner et al., 2009). Indeed, these shape-based associations may permeate to synesthetic associations throughout development, as in adult synesthetes similarly shaped letters and numbers tend to have similar synesthetic colors (Brang et al., 2011a; Watson et al., 2012). Building on this evidence, Brang et al. (2010) suggested that synesthetic associations between shapes and colors remain a vestigial relic in adult synesthetes. Indeed, while it is relatively uncommon for synesthetes to show conscious shape-color associations as adults, synesthetic colors are evoked in the temporal hierarchy prior to grapheme processing as revealed by MEG (Brang et al., 2010).

As the particular associations in the present study were assigned randomly, the majority of shape-color correspondences should be in disagreement with any given synesthete's latent and unconscious association. Indeed, one synesthete admitted that for one of the novel graphemes, a conscious synesthetic color emerged partway through the task and that this color differed from the association he was attempting to learn <sup>1</sup> . While speculative at present, we elaborate on the neonatal synesthesia hypothesis and suggest that synesthetic associations are present early in infancy as basic form-color associations that are tuned and biased over time into grapheme-color associations, eventually losing the initial conscious pairings between colors and shapes. Nevertheless, while these results suggest a latent influence of shapes on synesthetic colors, research demonstrates that colors are modulated by numerous factors including context (Ramachandran and Hubbard, 2001b; Dixon et al., 2006; Brang et al., 2008, 2011b), phonemic similarity (Asano and Yokosawa, 2012) and environment (Witthoft and Winawer, 2013).

Past studies in support of the "ideaesthesia" view of synesthesia, that synesthetic colors are evoked by semantic concepts as opposed to low-level stimulus properties, demonstrate that an arbitrary synesthetic color is able to transfer to a novel grapheme through semantic knowledge (Mroczko et al., 2009) and suggest that novel graphemes can take on any arbitrary color. Consistent with this finding, here synesthetes successfully learned arbitrary shape-color correspondences. Critically, however, this process was significantly more difficult in synesthetes compared to controls, suggesting latent color preferences in synesthetes conflicted with the enforced associations.

Grapheme-color synesthetes demonstrate enhanced memory recall for word lists that evoke synesthetic colors, line drawings that do not evoke synesthetic colors, and for memory of colors themselves (Yaro and Ward, 2007; Rothen and Meier, 2010; Gibson et al., 2012). However, memory enhancements in synesthesia are generally restricted to colors and visual stimuli and are not a generalized trait (for a review see Rothen et al., 2012). The present results further argue against a generalized memory enhancement in grapheme-color synesthetes suggesting they are not simply better at making arbitrary associations between complex shapes and colors. In addition, one possible explanation of why synesthetic colors transfer to novel symbols according to shape similarity is that this serves as a memory mnemonic, in that the color provides an extra cue leading to the remembrance of the shape of the grapheme. However, in these results synesthetes demonstrated impaired acquisition of the associations which would have utilized this approach.

One shortcoming of the present study is the absence of a within-group control demonstrating the specificity of this finding to confirm that grapheme-color synesthetes are not generally impaired in making novel associations. While these data cannot exclude such a possibility, future research may be able to rule out this possibility by examining synesthetes' ability to make colorcolor or shape-shape associations (in which we would expect no difference between synesthetes and controls). However, as mentioned above, synesthetes demonstrate better color memory in general (Yaro and Ward, 2007), which may instead facilitate synesthetes' ability to make color-color associations.

In sum, the current results suggest that synesthetic colors are biased in accordance with latent shape-color associations, and is consistent with views of this phenomenon emerging early in infancy with colors initially being evoked by shapes and basic forms. Future studies should seek to predict novel graphemecolor correspondences based on a synesthete's present set of number and letter-color associations to more directly bias subjects' ability to acquire these novel pairings. Given research suggesting a benefit of synesthetic colors on memory we expect that these novel associations will be better maintained if they are paired with synesthetic colors in line with those of their latent shape-color associations.

<sup>1</sup>Unfortunately we did not systematically ask our synesthetes whether this was a common phenomenon, necessitating additional research on this topic.

## **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fnhum.2013. 00717/abstract

#### **REFERENCES**


shapes. *Transl. Neurosci*. 3, 22–27. doi: 10.2478/s13380-012-0010-4

**Figure S1 | Individual subject plots showing synesthetes' (blue) and controls' (red) data points for each of the three analyses conducted for the Guess-and-Check paradigm (A–C) and for the Congruency paradigm.**

(D) Subject numbers are labeled along the x-axis. **∗∗***p <* **0.01**.


in grapheme-color synesthesia. *Nat. Neurosci.* 10, 792–797. doi: 10.1038/nn1906


*Psychon. Bull. Rev*. 19, 211–217. doi: 10.3758/s13423-011-0208-4


36, 155–193. doi: 10.1016/0010- 0277(90)90003-3

Yaro, C., and Ward, J. (2007). Searching for Shereshevskii: what is superior about the memory of synaesthetes? *Q. J. Exp. Psychol*. 60, 681–695. doi: 10.1080/174702106 00785208

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 July 2013; accepted: 09 October 2013; published online: 30 October 2013.*

*Citation: Brang D, Ghiam M and Ramachandran VS (2013) Impaired acquisition of novel grapheme-color correspondences in synesthesia. Front. Hum. Neurosci. 7:717. doi: 10.3389/ fnhum.2013.00717*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2013 Brang, Ghiam and Ramachandran. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Grapheme learning and grapheme-color synesthesia: toward a comprehensive model of grapheme-color association

## *Michiko Asano1,2,3\* and Kazuhiko Yokosawa4*

*<sup>1</sup> Department of Environment and Information Studies, Faculty of Environment and Information Studies, Keio University, Fujisawa, Japan*

*<sup>2</sup> Research Fellow of the Japan Society for the Promotion of Science, Tokyo, Japan*

*<sup>3</sup> Brain Science Institute, Tamagawa University, Tokyo, Japan*

*<sup>4</sup> Department of Psychology, Graduate School of Humanities and Sociology, The University of Tokyo, Tokyo, Japan*

#### *Edited by:*

*Beat Meier, University of Bern, Switzerland*

#### *Reviewed by:*

*Marcus R. Watson, The University of British Columbia, Canada Nathan Witthoft, Stanford University, USA*

#### *\*Correspondence:*

*Michiko Asano, Department of Environment and Information Studies, Faculty of Environment and Information Studies, Keio University, 5322 Endo, Fujisawa, Kanagawa, 252-0882, Japan e-mail: asano@sfc.keio.ac.jp*

Recent progress in grapheme-color synesthesia research has revealed that certain regularities, as well as individual differences, figure into grapheme-color associations. Although several factors are known to regulate grapheme-color associations, the impact of factors, including their interrelationships, on synesthesia remains unclear. We investigated determinants of synesthetic color for graphemes (characters, letters) of Hiragana, a phonetic script in the Japanese language, and the English alphabet. Results revealed that grapheme ordinality was the strongest predictor of synesthetic colors for Hiragana characters, followed by character sound, and visual shape. Ordinality and visual shapes also significantly predicted synesthetic colors for English alphabet letters, however, sounds did not. The relative impact of grapheme properties on grapheme-color associations and the differences between these two writing systems are accounted for by considering the way graphemes are processed in the brain and introduced during an individual's development. A new model is proposed which takes into account the developmental process of grapheme learning. The model provides comprehensive explanation of synesthetic grapheme-color association determination processes, including the differences across writing systems.

**Keywords: grapheme-color synesthesia, grapheme acquisition, grapheme discrimination, Japanese phonetic characters (Hiragana), language development, perceptual categorization and identification**

## **INTRODUCTION**

Grapheme-color synesthesia is a condition in which a visual letter or character (grapheme) induces a specific color sensation (e.g., the letter "r" may induce a concurrent sensation of "red"). Although it has been characterized as idiosyncratic, a number of regularities in the synesthetic experience have also been reported. Specifically, the factors that affect synesthetic grapheme-color correspondence are systematically associated with several grapheme properties: visual shape, sound, meaning or concepts, grapheme frequency, ordinality (positions in a grapheme sequence), and memory related to the graphemes (e.g., visual form: Brang et al., 2011; Watson et al., 2012; sound: Asano and Yokosawa, 2011, 2012; meaning or concepts: Rich et al., 2005; Asano and Yokosawa, 2012; grapheme frequency: Beeli et al., 2007; ordinality: Watson et al., 2012; memory: Witthoft and Winawer, 2006, 2013). What remains unclear is the relative impact of such various factors on synesthesia.

Some studies examined simultaneous influence of multiple grapheme properties on the same set of graphemes (Asano and Yokosawa, 2011, 2012; Watson et al., 2012). Asano and Yokosawa (2012) investigated the simultaneous influences of sound, visual shape, and meanings on synesthetic colors for Kanji characters (a logographic script used in the Japanese language) in Japanese grapheme-color synesthesia. Watson et al. (2012) also investigated

the influence of sound (letter names), visual shape, ordinality, and frequency on synesthetic colors for the modern English alphabet simultaneously in English grapheme-color synesthesia. Findings from these different studies reveal that several factors are concurrently involved in grapheme-color associations. However, to date the relative size of impact among the factors has not been the subject of specific investigation.

Results of some studies suggest that the magnitude of impact of each factor differs across writing systems, which poses a difficulty for resolving the puzzle of how several factors work together. For example, synesthetic colors for the English alphabet are usually determined by graphemes, not phonemes (e.g., the "c" in "cat" and in "cite" may both elicit the same color sensation, whereas initial letters of "site" and "cite" appear to elicit different colors; Simner, 2007). It is also reported that synesthetic color similarity across the English alphabet does not correlate with letter name similarity (e.g., although the name sounds of the letter "b" and "c" are similar, they are usually associated with dissimilar colors; Watson et al., 2012). In sharp contrast to English, synesthetic colors for Japanese phonetic scripts such as Hiragana and Katakana rely on sound quality, not on visual shapes of graphemes (Asano and Yokosawa, 2011). Hiragana and Katakana represent the same set of vowels or syllables (i.e., combinations of a consonant and a vowel) although their visual forms are dissimilar. Asano and Yokosawa (2011) reported that synesthetic color choices for Hiragana characters and those for their Katakana counterparts were remarkably consistent, indicating that color selection depended on character related sounds and not visual form (effects of ordinality and frequency were not investigated in this study).

Some researchers have claimed that synesthetic associations are acquired as graphemes are learned (Rich et al., 2005; Witthoft and Winawer, 2006, 2013; Wagner and Dobkins, 2011; Asano and Yokosawa, 2012; Watson et al., 2012). The relative size of factor impact may reflect the way in which graphemes were associated with colors in the brain of a synesthete in his/her childhood, when s/he was learning graphemes. Grapheme learning consists of visual shape discrimination of graphemes and the association of graphemes with phonemes (in the case of phonetic scripts including the English alphabet) or with phonemes and meanings (in the case of logographic scripts). Watson et al. (2012) showed that letter shape and ordinality were linked to hue, and letter frequency was linked to luminance of synesthetic colors for the English alphabet in adult grapheme-color synesthetes. Based on such findings, they conjectured that associating letter shapes and identities with hue aided letter learning (categorical discrimination of letters), and that letter frequency, which varies along a continuum, mapped naturally to luminance in childhood.

Several causal factors for the association of colors with graphemes during grapheme learning have been proposed in various studies. For instance, Rich et al. (2005) raised the possibility that colors are linked to items forming conventional sequences (e.g., alphabets, numbers, days of the week, months of the year) before the acquisition of graphemes (letters, numerals); these links then are generalized to graphemes and other items that are not part of such sequences (e.g., words). Likewise, Asano and Yokosawa (2012) suggested the possibility that grapheme-based synesthesia begins as phoneme synesthesia (i.e., colors are linked to phonemes prior to grapheme acquisition). According to the observation that characters representing the same sounds tended to elicit similar colors in Japanese grapheme-color synesthesia regardless of their visual shapes, they surmised that, in some cases, the colors linked to phonemes before grapheme acquisition are subsequently generalized to graphemes via phonology. Some related evidence for early associations of simple visual shapes with colors comes from an infant study by Wagner and Dobkins (2011), in which they demonstrated that the presence of particular shapes influences color preferences in typical 2- and 3-month-olds (but not in 8-month-olds or adults). Although further research is required to clarify the relationships between such "neonatal synesthesia" and grapheme-color synesthesia, this study suggests that shape-color linkages may be formed in the brain before grapheme acquisition. External input can also be a source of synesthetic colors. Witthoft and Winawer showed that some synesthetes had startlingly similar color-grapheme pairings traceable to childhood toys containing colored letters, such as refrigerator magnets (Witthoft and Winawer, 2006, 2013). All these studies suggest the importance of developmental perspectives in achieving a complete understanding of grapheme-color association process; however, the interrelationships among the factors raised above are not still clear.

Considering previous developmental research with graphemecolor synesthesia, in the present study we propose a model that offers a comprehensive explanation of synesthetic graphemecolor association determination processes, including the differences across writing systems. Five main claims of this model are: (1) synesthetic associations are acquired as graphemes are learned. (2) Children who are learning graphemes use information from various feature domains to differentiate among graphemes. (3) Linkages between color representations and representations of features in some domains (e.g., ordinality in conventional sequences, sounds, and visual shapes) are formed in the brain before grapheme acquisition. (4) When a grapheme is learned, (one of the) color representations associated with representations of the grapheme's features is/are mapped onto the grapheme. The color for the representation of feature which makes the largest contribution in discriminating the grapheme from others is the color most likely to be ultimately associated with the grapheme. (5) The resulting synesthetic color highlights the discriminating feature(s) of the grapheme. This facilitates grapheme discrimination and in turn grapheme learning.

#### **TWO EXAMPLES: ENGLISH LETTERS AND JAPANESE CHARACTERS**

Schematic illustrations of the proposed model appear in **Figure 1** for each of two example language systems. First, consider a model for English alphabets, shown schematically in **Figure 1A**. In the English alphabet, graphemes (letters) form a sequence. The ordinality domain allows for identifying one grapheme with one ordinal representation of a character because each grapheme is assigned only one ordinal number (e.g., first, second, third).

In the visual shape domain, correspondences between graphemes and representations are rather complex compared to those in the ordinality domain. Assuming that visual shapes of graphemes are processed in units of basic distinctive visual features (cf. Gibson, 1969), one grapheme should be linked to several basic visual feature representations, because graphemes usually combine several basic visual features (except for very simple graphemes like "I" and "O," which may be linked to only one basic visual feature representation).

For phonological domains, a highly complex relationship exists between graphemes and phonological representations in the case of the English alphabet. There are two kinds of phonological information associated with a grapheme: the name of grapheme and its pronunciation. For instance, the letter "b" has the grapheme name of "bee" and it is pronounced as /b/. In short, learning an English grapheme requires acquiring two kinds of associations of phonological information with a single grapheme. A typical English alphabet letter has several pronunciations, and the pronunciation of the letter name is usually different from any of the letter pronunciations (e.g., pronunciation of the letter "c" can be either /s/, /k/, or "ch" in "chance"; the pronunciation of the name of the letter "c" is /see/, which is different from any of the letter pronunciations). In addition to the one-to-many correspondences between graphemes andphonemes, some English phonemes are represented by several graphemes; for example, the phoneme /s/ is represented by both letters "c" and "s," and the phoneme /z/ is represented by both "s" and "z" (i.e., many-to-one correspondences).

To summarize, there are simple (i.e., one-to-one) ordinalitygrapheme correspondences, relatively complex shape-grapheme correspondences, and highly complex phoneme-grapheme correspondences in the English alphabet. When the corresponding relations between grapheme and feature representations in a certain domain are simple (e.g., when one-to-one relationship exists among graphemes and feature representations), mapping the corresponding representations (and the color associated with each of the representations) to the graphemes can provide an additional source of information available for grapheme discrimination. Complex correspondences between graphemes and representations of a given domain, on the other hand, can reduce the influence of that feature domain on grapheme-color associations. This is because such a feature domain is likely to contribute little to a differentiation among graphemes. Thus, the synesthetic color that is finally associated with a grapheme is determined. Taking these factors into account, our model predicts that ordinality should have the greatest impact on synesthetic colors for the English alphabet, followed by visual shapes. The impact of phonology on synesthesia should be very small because graphemes are arbitrarily associated with phonemes in the English alphabet. This prediction is at least partly in accordance with results of previous studies that report the influence of visual shape (e.g., Brang et al., 2011), or of ordinality and visual shape but not of sound (Watson et al., 2012; see also Simner, 2007) on grapheme-color synesthesia for the English alphabet (This is "at least" because no previous study examined the relative size of impact of these factors).

Consider a second example which involves Japanese Hiragana characters (see **Figure 1B**). Our model is best characterized by contrasting cases of graphemes of different writing systems with different characteristics. As noted above, Hiragana is a phonetic script that represents a set of vowels and syllables (see Table S1 in Supplementary Materials). Hiragana characters are the graphemes that most native Japanese speakers learn first (usually at around the ages of 4–6 years; see (Asano and Yokosawa, 2012), for detailed descriptions of scripts used in the Japanese language). As in the case of the English alphabet, there are simple correspondences between graphemes (characters) and ordinal representations in Hiragana characters, as each grapheme is assigned only one ordinal number. In the domain of visual shape, as in the case of the English alphabet, correspondences between graphemes and representations are somewhat complex compared to those in the ordinality domain. This is because graphemes are made of combinations of several basic visual features.

In the phonological domain, however, Hiragana characters are very different from English alphabet letters. In the case of Hiragana characters, each character has only one pronunciation and it is the same as the pronunciation of the character name (e.g., the name of the first character in the Hiragana sequence is "a" (/a/) and the character is also pronounced as /a/). Consequently, there is a one-to-one relationship between graphemes and phonemes1 in Hiragana characters. This reasoning leads to the prediction that phonology and ordinality are strong determinants of synesthetic colors and that the visual shapes of characters should have less impact in grapheme-color synesthesia for Hiragana characters. This prediction is partly consistent with Asano and Yokosawa (2011). In their study synesthetic colors for Hiragana characters were found to be elicited by sound qualities and not by visual shapes. Thus, the proposed model has the ability to explain the differences in determinants of grapheme-color association across writing systems.

#### **A NEW MODEL**

The present model goes beyond simply providing a theoretical backbone to existing observations. Using this model, we can predict the relative impact of possible determinants of synesthetic colors, such as ordinality, phonology, and visual shapes, and draw a comprehensive picture of grapheme-color association process in synesthesia. The purpose of this study is to test the validity of this model by examining its predictions of the relative impacts of several factors on synesthetic color selection. To this end, we predicted the interrelationships of several possible determinants of synesthetic colors in Japanese Hiragana characters; in addition these predictions were tested in a behavioral experiment using synesthetes whose native language is Japanese. We chose Hiragana characters because, according to model predictions, there are more grapheme properties that can have significant (observable) impact on grapheme color associations than in the case of English alphabet letters. This will enable a more efficient test of the model.

<sup>1</sup>More precisely, one Hiragana character is linked to either one phoneme or one pair of phonemes, because Hiragana script is a syllabary representing vowels or syllables (i.e., combinations of a consonant and a vowel). Nonetheless, we call this "one-to-one relationship between graphemes and phonemes" in this article for convenience. The point here is that grapheme-phoneme correspondence is transparent in Hiragana characters.

We chose to evaluate Hiragana characters rather than Katakana characters (which are also phonetic characters in the Japanese language with similar properties as Hiragana characters) because Hiragana characters are the graphemes that most native Japanese speakers learn first (analogously, letters of the English alphabet are graphemes learned first by most native English speakers). Katakana characters are usually learned after Hiragana characters (see Asano and Yokosawa, 2011, 2012).

The model leads to the following central hypothesis: Synesthetic colors for Hiragana characters are greatly influenced by ordinality and sounds (phonemes), whereas synesthetic colors for the English alphabet are greatly influenced by ordinality, but not by sounds (phonemes). In both cases the model predicts that synesthesia is weakly affected by visual shapes.

The rationale behind this hypothesis holds that among the three feature domains of interest (sound, ordinality, shapes), sounds should have the weakest impact on synesthetic colors with the English alphabet because one grapheme is usually arbitrarily associated with several phoneme representations. In addition to synesthetic colors for Hiragana characters, we also examined those for English alphabet letters in Japanese synesthetes for a further test of the model. Although the English alphabet is not a writing system for their mother tongue (i.e., Japanese), most Japanese people learn it at around the ages of 9–10 years in elementary school and have no difficulty in reading and writing English alphabet letters. The English alphabet is first introduced to Japanese children as a sequence (i.e., the ABC sequence) and then they learn to represent sounds of Japanese words using the English alphabet (referred to as Roma-ji). At that time, children notice that the pronunciations of letter names and that of letters are not the same in the English alphabet as well as that some letters (e.g., L, Q, X) are not used to represent Japanese sounds because there are no corresponding sounds in the Japanese language. All of these considerations figured into our main prediction that synesthetic colors for the English alphabet will be greatly influenced by ordinality, and less affected by visual shapes, and least affected by sounds even in Japanese synesthetes.

To test model predictions, we used multiple regression analyses to assess the relative impact of ordinality, sound, visual shape, and grapheme familiarity on synesthetic color choice for Hiragana characters and English alphabet letters. We included grapheme familiarity in addition to the other three predictors, although familiarity is not referred in the model (cf. **Figure 1**). This is because there have been a few reports of some influence of grapheme frequency on synesthetic colors (e.g., Beeli et al., 2007; Watson et al., 2012). Grapheme familiarity is closely related to grapheme frequency. These two factors differ in that familiarity refers to subjective familiarity ratings of graphemes whereas grapheme frequency refers to grapheme norms in a defined corpus of publications, such as newspapers. We used subjective familiarity instead of frequency because it should more validly reflect processing efficiencies of the characters/letters in the brain of young children, who have not yet been exposed to the published literature.

We employed the idea of second-order similarity mappings in grapheme-color synesthesia proposed by Watson et al. (2012). Second-order mappings refer to correlations between differences in values in one domain of grapheme features and differences in synesthetic colors of graphemes (e.g., more visually similar letters tend to be associated with more similar colors). This is contrasted to the idea of first-order mappings, which are correlations between absolute (non-relational) properties of a letter and dimensions of synesthetic color (e.g., the letter "y" tends to be yellow, because "y" is for the word "yellow"). Second-order mapping analysis is suitable for examining the influence of the four grapheme properties on synesthetic colors, ordinality, phonology, visual shapes, and grapheme familiarity, because all the four of these properties can be described relationally (i.e., more similar/different). Examining relations involving differences between graphemes and their assigned colors allowed us to directly compare and contrast multiple influences on synesthetic graphemecolor associations. Whereas Watson and his colleagues examined the independent effects of grapheme properties (e.g., letter shape, frequency, and ordinality) on grapheme-color associations in synesthesia, in this study, we focused on the *relative* impact of these properties. The aim was to test our hypothesis that synesthetic colors are associated with graphemes during grapheme acquisition, and that the color for the representation of the feature which makes a *relatively large* contribution in discriminating the grapheme from others is the color most likely to be ultimately associated with the grapheme. We believe we can provide a comprehensive picture of grapheme-color association in synesthesia by focusing on the relative impact of grapheme properties and considering the ways in which graphemes are introduced to children during development.

## **MATERIALS AND METHODS**

## **PARTICIPANTS**

Seventeen Japanese grapheme-color synesthetes (16 female, one male, mean age = 25.0 years, *SD* = 7*.*2 years), whose first language is Japanese, participated in this study. All reported experiencing synesthetic colors when viewing Hiragana characters and English alphabet letters, as well when viewing Katakana and Kanji characters and Arabic numerals (i.e., digits). All reported that auditory input of linguistic sounds does not elicit synesthetic experiences (unless converted to graphemes). Informed consent was obtained from all participants of this study after the nature and possible consequences of the studies were explained. The rights of the participants were protected. All the experimental procedures had been approved by the Ethical Committee of Department of Psychology, Graduate School of Humanities and Sociology, The University of Tokyo in Japan.

To establish the genuineness of participating synesthetes, we also recruited six female non-synesthetic control participants; mean age was 22.3 years (*SD* = 2*.*6 years). All were undergraduate or graduate students from universities in Japan and were native speakers of Japanese.

## **APPARATUS**

Characters and a color palette were displayed on a monitor (Mitsubishi Diamondtron M2 RDF223G). One hundred and thirty-eight HTML named colors (colors supported by major web browsers) were in the palette (Asano and Yokosawa, 2011, 2012). We chose this color set because a palette should consist of common and mutually distinguishable colors. Each of the 138 colors has its own name, therefore we regard the palette as a fine category set of colors which is used conventionally and universally. The palette incorporated a variety of colors while reducing the requisite effort in selecting colors (compared to the use of an extensive color palette as in Eagleman et al. (2007), which has 256 × 256 × 256 colors). Note, however, that use of the 138 color palette may reduce the sensitivity of the experimental results to the real effects of various factors on color-grapheme synesthesia. Color coordinates in the CIE L∗a∗b∗ systems, used in analyses of the results, were converted from the CIE xyY coordinates of presented colors; all were measured using Topcon BM-7 luminance colorimeter. We adopted the CIE L∗a∗b∗ system because it was designed in such a way that mathematical differences in all color ranges correspond to perceived color differences (Kaatsch et al., 1993). We further converted the color coordinates in the CIE L∗a∗b∗ system to coordinates in the CIE L∗C∗h system in order to obtain hue, saturation, and luminance values.

#### **STIMULI**

Hiragana characters: Among the entire set of 71 Hiragana characters, all 46 basic characters were used as stimuli<sup>2</sup> . Five of these were symbols for single vowels (/a/, /i/, /u/, /e/, and /o/). Of the remaining characters, 40 were symbols for syllables that is, combinations of a consonant (/k/, /s/, /t/, /n/, /h/, /m/, /y/, /r/, /w/) and one of the five vowels (no characters for syllables /yi/, /ye/, /wi/, /wu/, /we/ exist in the Japanese language). The remaining character represents the sound of /n/, which has no vowel.

English alphabet letters: The entire set of 26 English uppercase alphabet letters were employed as stimuli.

#### **PROCEDURE**

While seated in a darkened room, each synesthete used a computer mouse to select a palette color corresponding to one of the stimulus characters/letters. Synesthetes were told to select a color closest to their synesthetic colors. They were also told to select the color black if a presented character elicited a sensation of the color black or no synesthetic color. The Hiragana characters and English alphabet letters were presented (for color selection) in a randomized order in respectively, separate blocks in a session. No characters/letters were repeated within a session.

Two temporally separated experimental sessions were presented to participants in order to test the consistency of their color selections over time (Eagleman et al., 2007). The stimuli were the same, but the order of character/letter appearance differed randomly between the two sessions. Sessions were separated by intervals of at least 3 weeks (mean interval = 10.1 weeks, *SD* = 11*.*2 weeks). The averages of color distances (Euclidean distance in the CIE L∗a∗b∗) for colors selected for a given character/letter in the first and second session, calculated and averaged separately for the 46 Hiragana characters and the 26 English alphabet letters, were very small; 19.7 (*SE* = 1*.*4, range = 10.3–31.3) for Hiragana characters, and 16.3 (*SE* = 1*.*2, range = 6.8–25.3) for English alphabet letters. These values are strikingly small given that the average color distance for random color pairs from the 138 colors in the palette (i.e., chance level) was 67.9 (*SD* = 31*.*8) and analogous values obtained from six non-synesthetic controls, who engaged in the same task as synesthetic participants with 2 weeks intervals between the two sessions, were 62.4 (*SE* = 7*.*2, range = 33.3–86.2) for Hiragana characters and 65.8 (*SE* = 7*.*7, range = 29.2–86.1) for English alphabets. An unequal variance *t*-test (Welch's method, two-tailed) revealed that the mean color distances across sessions for the synesthete group were significantly smaller than those for the control group, *t(*5*)* = 5*.*34 for Hiragana characters and *t(*5*)* = 5*.*83 for English alphabet letters, both *p*s *<* 0.01, Bonferroni corrected. These results clearly demonstrate that participants engaged in this study were genuine synesthetes. Data from the first session was used for the following analyses.

## **DATA PREPARATION**

Using the method proposed by Watson et al. (2012), we explored second-order similarity mappings. In addition to color distance (Euclidean distance in CIE L∗a∗b∗ color space), we computed luminance distance, saturation distance (difference of C∗ values in the CIE L∗C∗h color coordinate) and hue distance (difference of *h*-values in the CIE L∗C∗h color coordinate) between two graphemes separately as color difference measures. Four dependent measures of grapheme difference/similarity were compared with each of the synesthetic color difference measures: ordinality difference, phonological similarity, visual shape similarity, and grapheme familiarity difference. Ordinality difference refers to a difference between the positions in the Hiragana/alphabet sequence of two characters/letters, divided by their sum. Phonological similarity refers to the number of shared phonemes (consonants and vowels) in names of two characters/letters. Graphemes names are exactly the same as the pronunciation in the case of Hiragana, but this is not in the case for letters in the English alphabet. Visual similarity refers to subjective comparison ratings of the similarities of two characters/letters in Hiragana (Kawakami and Tsuji, 2012) and the English alphabet (Boles and Clifford, 1989), both rated on a five-point scale from 1 (low similarity) to 5 (high similarity). Character/letter familiarity differences refer to differences between the subjective familiarity ratings of two Hiragana characters/English alphabet letters, both rated by adult native Japanese speakers in a Japanese language environment on a seven-point scale from 1 (low familiarity) to 7 (high familiarity) (Amano and Kondo, 1999).

There were 1035 and 325 possible Hiragana character and English alphabet letter pairs, respectively (not including doubles of the same character/letter). We computed separate values for color distance, luminance distance, saturation distance, and hue distance for each of these pairs. All subsequent analyses were performed after binning the 1035 Hiragana character pairs into 69 bins that each included 15 character pairs. For 325 English alphabet letter pairs 65 bins were used, each included 5 letter

<sup>2</sup>The remaining 25 non-basic characters are so-called "dakuon" and "handakuon" characters in the Japanese language. These characters were not used as stimuli in this study because their positions in the Hiragana sequence are not well established and subject to change depending on circumstances (e.g., the index in a dictionary, word sorting functions on a computer). The basic and non-basic Hiragana characters are usually appear in separate tables in educational materials.

pairs <sup>3</sup> (see Watson et al., 2012, for the original method). Bins were determined by the mean color distance of each character/letter pair across all 17 synesthetes, such that the first bin of the Hiragana character pairs and English alphabet pairs, respectively, contained pairs whose two characters/letters were, on average, most similar in color.

3The number of pairs included in a bin differed between Hiragana characters (15 pairs) and English alphabet letters (5 pairs). This was to match the numbers of items used in each multiple regression analysis between the two scripts as closely as possible. The overall results for Hiragana characters were almost the same between when 5 pairs were included in a bin (i.e., 207 bins were used for multiple regression analyses. See Tables S2, S3 in Supplementary Materials for detailed results) and when 15 pairs were included in a bin (see **Table 2** in the text).

## **RESULTS**

#### **HIRAGANA CHARACTERS**

Correlations among variables for Hiragana characters are shown in **Table 1**. To test for multicollinearity among independent variables, we examined the variance inflation factor (VIF). The VIF values were all below 1.5, indicating multi-collinearity was not an issue. See Figure S1 in Supplementary Materials for a visualization of distance matrices of each color/grapheme property measure for Hiragana characters.

As shown in **Table 2**, a multiple regression model using the four independent variables (ordinality difference, phonological similarity, shape similarity, and familiarity difference) as predictor variables for color distance in synesthetic colors for Hiragana characters [*R*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*79, *<sup>F</sup>(*4*,* <sup>64</sup>*)* <sup>=</sup> <sup>59</sup>*.*66, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*01]. This analysis

### **Table 1 | Correlations among the variables for Hiragana characters (***N* **= 69).**


*\*\*p < 0.01.*

#### **Table 2 | Results of multiple regression analyses for Hiragana character pairs.**


*SE, standard error.*

also revealed that ordinality difference was the strongest predictor of synesthetic color distance (Beta = 0.394, *p <* 0*.*01), followed by phonological similarity (Beta = −0*.*333, *p <* 0*.*01), shape similarity (Beta = −0*.*225, *p <* 0*.*01), and familiarity difference (Beta = 0.213, *p <* 0*.*01). Overall, this analysis supports our hypothesis, which predicted a large impact of ordinality and phonology, and a relatively small impact of visual shape on synesthetic colors for Hiragana characters.

We also conducted multiple regression analyses to predict luminance, saturation, and hue distances separately using the same four independent variables (**Table 2**). A regression model using the four independent variables to predict luminance distance in synesthetic colors for Hiragana characters [*R*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*51, *F(*4*,* <sup>64</sup>*)* = 16*.*90, *p <* 0*.*01] also revealed that ordinality difference was the strongest predictor (Beta = 0.386, *p <* 0*.*01), followed by shape similarity (Beta = −0*.*276, *p <* 0*.*01) and phonological similarity (Beta = −0*.*268, *p <* 0*.*05). Luminance distance was not significantly predicted by familiarity difference (Beta = −0*.*036, *p* = 0*.*73). A similar trend was observed in the multiple regression analysis for saturation distance [*R*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*73, *F(*4*,* <sup>64</sup>*)* = 42*.*27, *p <* 0*.*01]; ordinality difference was the strongest predictor (Beta = 0.312, *p <* 0*.*01), followed by shape similarity (Beta = −0*.*308, *p <* 0*.*01) and phonological similarity (Beta = −0*.*269, *p <* 0*.*01). Familiarity difference also significantly predicted saturation distance (Beta = −0*.*247, *p <* 0*.*01). A regression model to predict hue distance [*R*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*74, *<sup>F</sup>(*4*,* <sup>64</sup>*)* <sup>=</sup> <sup>46</sup>*.*18, *p <* 0*.*01] showed hue distance was significantly predicted only by phonological similarity (Beta = −0*.*427, *p <* 0*.*01) and ordinality difference (Beta = 0.385, *p <* 0*.*01). In addition to this, shape similarity and familiarity difference each levied only a marginally significant effect on predicted hue distance (Beta = −0*.*149, *p* = 0*.*05; Beta = 0.141, *p* = 0*.*07, respectively).

#### **ENGLISH ALPHABET LETTERS**

Correlations among variables for English alphabet letters are shown in **Table 3**. The VIF values were all below 1.1, indicating multicollinearity was not an issue. See Figure S2 in Supplementary Materials for visualized distance matrices of each color/grapheme property measures for English alphabetical letters.

**Table 4** shows the results of a multiple regression model using the four independent variables as predictors for color distance in synesthetic colors for English alphabets [*R*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*27, *<sup>F</sup>(*4*,* <sup>60</sup>*)* <sup>=</sup> 5*.*50, *p <* 0*.*01]. It reveals that ordinality difference was the only statistically significant predictor (Beta = 0.393, *p <* 0*.*01); shape similarity had a marginally significant effect on predicted hue distance (Beta = −0*.*201, *p* = 0*.*10).

We also conducted multiple regression analyses to predict luminance, saturation, and hue distances of synesthetic colors for English alphabet letters, respectively, with the four independent variables (**Table 4**); however, multiple regression models to predict luminance distance and saturation distance were not statistically significant [*R*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*13, *<sup>F</sup>(*4*,* <sup>60</sup>*)* <sup>=</sup> <sup>2</sup>*.*24, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*08; *<sup>R</sup>*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*11, *<sup>F</sup>(*4*,* <sup>60</sup>*)* <sup>=</sup> <sup>1</sup>*.*76, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*15, respectively]. The regression model for hue distance was significant [*R*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*16, *<sup>F</sup>(*4*,* <sup>60</sup>*)* <sup>=</sup> 2*.*76, *p <* 0*.*05], and it was shown that shape similarity significantly predicted hue distance (Beta = −0*.*280, *p <* 0*.*05); the ordinality difference had only a marginally significant effect on predicted hue distance (Beta = 0.217, *p* = 0*.*08).

#### **DISCUSSION**

#### **SUMMARY OF THE FINDINGS**

The results for Hiragana characters, in general, supported our predictions. As predicted, a multiple regression analysis revealed that ordinality difference and phonological similarity were strong predictors (Beta = 0.394, −0.333, respectively) of synesthetic color distances for Hiragana characters. The impact of visual shape similarity, although also statistically significant, was relatively small (Beta = 0.225). The impact of both phonology and ordinality was manifest in the dimension of hue of synesthetic colors, whereas the impact of visual shape and grapheme familiarity on the hue dimension was relatively small (statistically marginally significant). Ordinality difference was not only the strongest predictor of color distance; it was also dominant in predicting luminance, and saturation. The impact of visual shape similarity was comparable to that of phonological similarity in the luminance dimension and greater than that of phonological similarity in the saturation dimension, both of which were slightly different from our prediction. Grapheme familiarity difference affected color distance and saturation of synesthetic colors significantly. Thus, we observed significant influence of visual shape on synesthetic colors. This finding differs from that reported by Asano and Yokosawa (2011), who found no effect of visual shape on synesthetic colors for Hiragana characters. This inconsistency can be explained as follows. Asano and Yokosawa (2011) concluded that synesthetic color selection for Hiragana characters


*\*p < 0.05, \*\*p < 0.01.*


**Table 4 | Results of multiple regression analyses for English alphabet letter pairs.**

*SE, standard error.*

did not depend on visual shape based on the fact that synesthetic colors for two characters sharing the same sounds, namely a Hiragana character and its Katakana counterpart, were found to be remarkably consistent regardless of the differences in their shapes. That is, Asano and Yokosawa compared visual shapes of two characters from respectively different sets of graphemes. The presence of visual shape effects in the present results suggests that it is *relative* shape similarity within a set of graphemes that affects synesthetic color similarity rather than the *absolute* similarity/difference of visual images. In other words, relations between graphemes are measured independently across different sets of graphemes.

The results for English alphabet letters were also generally in accordance with the predictions of the model proposed in this study. We predicted a strong influence of ordinality for these stimuli and a weaker influence of visual shape. Finally, phonology was predicted to have the weakest influence on synesthetic colors for English alphabet letters. The results showed that statistically the ordinality difference served as a significant predictor of color distance and that there was also a marginally significant impact of shape similarity; however, there was no significant impact of phonology. Moreover, visual shape similarity proved to be a statistically significant predictor of hue and there was also a marginally significant impact of ordinality difference; again there was no significant impact of phonology. However, the explanatory powers of the models for English alphabet letters were generally small (at most *<sup>R</sup>*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*27, in the case of color distance) compared to those for Hiragana characters (*R*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*51∼0*.*79).

#### **CONGRUENCE OF CATEGORICAL vs. CONTINUOUS PROPERTIES BETWEEN GRAPHEMES AND COLORS**

Watson et al. (2012) conjectured that a congruency factor can explain certain selective effects observed in second-order relations among grapheme-color pairs. They examined various associations between synesthetic colors and English alphabet letters (in synesthetes who were probably English speakers, although their native language was not specified in Watson et al., 2012). Of most interest, they proposed that grapheme properties that are processed categorically (e.g., letter shape) are mapped to a congruent, i.e., categorically perceived, color dimension (i.e., hue). Similarly, they proposed that grapheme properties that are processed continuously (e.g., frequency) are mapped to color dimensions that are perceived as continuums (i.e., luminance and saturation). In their study, shape difference (calculated based on 11-dimensional distinctive letter features proposed by Gibson, 1969) was found to correlate significantly with color and hue distance, and frequency difference significantly correlated with luminance distance; in addition, the ordinality difference significantly correlated with color and hue distance. They explained the correlation between ordinality difference and hue by assuming that ordinality may be processed categorically, although it varies continuously, because it is used as a rough index of the order of learning of individual letters.

Although Watson et al. (2012) examined synesthesia associated with English letters, their interpretations offer some generality which may facilitate understanding findings with Hiragana characters in the present study. In terms of congruence of categorical vs. continuous properties, we observed a significant influence of grapheme familiarity, which varies continuously, on color distance and saturation distance, both of which also (at least in some part) vary continuously, among Hiragana characters. Phonology, which is a strictly categorical entity, was the strongest predictor of hue distance, and it is often perceived categorically. However, ordinality strongly affected all the dimensions of synesthetic colors, both categorical (i.e., hue, and color distance in some part) and non-categorical (i.e., luminance, saturation, and color distance in some part). The latter outcome may suggest that, for some reasons, both categorical and continuous aspects of ordinality are reflected to synesthetic colors for Hiragana characters. Part of the reason may be the fact that there are more Hiragana characters (46 basic characters, 71 characters in total) than letters in the English alphabet (26 characters). The number of Hiragana characters may be sufficiently large such that it renders it difficult to process all the characters categorically (e.g., to use ordinality as a rough index of the order of learning of each grapheme); thus continuous aspects of ordinality may be relatively emphasized.

#### **WHICH ASPECTS OF VISUAL SHAPE MATTER?**

The results of Watson et al. (2012) and those of the present study differ in a few respects. In particular visual shape influenced hue of synesthetic colors for English alphabet letters in the study of Watson et al. (suggesting that visual shapes are processed categorically). By contrast, in our study visual shape affected luminance, saturation, and color distance among synesthetic colors for Hiragana characters (suggesting that visual shapes are processed continuously), and it also had a marginally significant effect on hue distance (suggesting that visual shapes are processed categorically). This may be due to the differences between the two sets of graphemes. Because the number of graphemes is greater in the Hiragana syllabary than it is in the English alphabet, and because Hiragana characters are generally visually more complex than the English alphabet, some aspects of visual shapes of Hiragana characters may be processed more quantitatively as compared with processing in the English alphabet. For example, not only the presence/absence of basic visual features (i.e., categorical, qualitative properties) but also properties such as relative length of components and number of strokes (i.e., quantitative properties) may have strong impact on Hiragana character discrimination.

The results for English alphabet letters in the present study are similar to those reported by Watson et al. (2012) using English speakers. For example, visual shape in both studies affected hues. However, we must acknowledge that the results differ in some respects. As visual shape similarity/difference measure(s), we used subject letter similarity ratings from Boles and Clifford (1989) in the present study, whereas Watson et al. (2012) used several different measures including both the letter similarity ratings (from Boles and Clifford, 1989) and the shape difference calculated based on 11-dimensional distinctive letter features proposed by Gibson (1969). Thus, the similarity ratings from Boles and Clifford (1989) were used in both studies. However, for some reason, this predictor significantly affected hues in the present study but not in the Watson et al. (2012) study. Instead, in Watson et al. (2012), as noted above, the shape difference measure, calculated using the basic letter shape features from Gibson (1969) significantly affected hues.

To assess the impact of the shape difference measure (based on Gibson, 1969) on synesthetic colors for English alphabet letters reported by Japanese synesthetes in the present study, we conducted additional multiple regression analyses. We sought to predict color, luminance, saturation, and hue distance in synesthetic colors for English alphabet letters found in present study using ordinality difference, phonological similarity, familiarity, and the visual shape difference (calculated based on Gibson, 1969, instead of subjective shape similarity provided by Boles and Clifford, 1989) as predictors <sup>4</sup> . The overall results were similar to those of the multiple regression analyses in the present study (see Results) in which the Boles and Clifford's measure was used as the visual shape similarity measure: multiple regression models to predict luminance distance and saturation distance were not statistically significant [*R*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*13, *<sup>F</sup>(*4*,* <sup>60</sup>*)* <sup>=</sup> <sup>2</sup>*.*18, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*08; *<sup>R</sup>*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*12, *<sup>F</sup>(*4*,* <sup>60</sup>*)* <sup>=</sup> <sup>1</sup>*.*98, *<sup>p</sup>* <sup>=</sup> <sup>0</sup>*.*11, respectively]. The regression model for hue distance and color distance were significant [*R*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*28, *<sup>F</sup>(*4*,* <sup>60</sup>*)* <sup>=</sup> <sup>5</sup>*.*97, *<sup>p</sup> <sup>&</sup>lt;* <sup>0</sup>*.*01; *<sup>R</sup>*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*32, *<sup>F</sup>(*4*,* <sup>60</sup>*)* <sup>=</sup> <sup>7</sup>*.*00, *p <* 0*.*01, respectively], and it was shown that hue distance was significantly predicted by the shape difference (Beta = 0.462, *p <* 0*.*01), and color distance was significantly predicted by ordinality difference and the shape difference (Beta = 0.394, 0.303, respectively, both *p*s *<* 0.01). Note that the explanatory powers of the models are greater when the Gibson's measure was used than when the Boles and Clifford's measure was used (hue distance: Boles and Cliffor's measure *<sup>R</sup>*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*16, Gibson's measure *<sup>R</sup>*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*28; color distance: Boles and Cliffor's measure *<sup>R</sup>*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*27, Gibson's measure *<sup>R</sup>*<sup>2</sup> <sup>=</sup> <sup>0</sup>*.*32), which may be in accordance with Watson et al. (2012) in which significant effects of the Gibson's measure but not Boles and Clifford's measure were observed. Since Gibson (1969) considered neurophysiological and developmental observations in selecting the distinctive letter features while the Boles and Clifford's measure, which consists of ratings on subjective letter similarity by adults and may reflect various information associated to the letters such as knowledge, it may be these types of neurophysiological processing of basic visual properties in childhood that affect synesthetic color associations for English alphabet letters. This is in accordance with the model proposed in this study.

#### **SYNESTHETIC COLORS AS "DISCRIMINATING MARKERS" OF GRAPHEMES DURING LEARNING GRAPHEMES**

The model we have proposed explains the synesthetic color for Hiragana characters quite well (79 and 74% of the variance was predicted by the four variables in the case of color distance and hue distance, respectively). However, its explanatory power declined for the English alphabet letters in this study (only 27 and 16% of the variance was predicted in the case of color distance and hue distance, respectively). Since the number of grapheme pairs included in a bin in the second-order mapping analyses differed between Hiragana characters and English alphabet letters (15 and 5 pairs, respectively. See footnote 3 for more information), it may be difficult to make direct comparisons of explanatory powers of the models between Hiragana characters and English alphabet letters. Even taking this into account, the model proposed in this study still appears to explain the synesthetic color for Hiragana characters better than it explains synesthetic color for English alphabet letters; the results showed that the phonological similarity had a strong effect in Hiragana characters but not in English

<sup>4</sup>Unfortunately, visual shape difference measures for Hiragana characters which are analogous to the Gibson's measure do not exist. Hiragana characters are visually too complicated to classify based on the existence/absence of simple basic visual features. Therefore we could not conduct analogous analyses for Hiragana characters in this study.

alphabets, which means that the model revealed more predictors in the case of Hiragana characters.

One possible reason for the low explanatory power for English alphabet letters in this study is that the English alphabet is not the writing system of the native language of our participants and they are not very familiar with its letters. According to a corpus of a major newspaper, the mean frequency of 26 English alphabet letters is roughly 1/42 of the mean frequency of 46 basic Hiragana characters (Amano and Kondo, 2000). The lack of influence of letter familiarity on synesthetic colors for English alphabet letters in this study supports this idea (i.e., letter familiarity was too low in general to affect synesthetic color choices). Another possible reason for this result is that the synesthetes who participated in this study had already acquired several sets of graphemes (e.g., Hiragana, Katakana, and some Kanji characters) prior to acquiring letters in the English alphabet. As previous studies have shown, synesthetic colors for graphemes acquired early in life can transfer to those for graphemes acquired later in life (Mills et al., 2002, 2009; Witthoft and Winawer, 2006; Asano and Yokosawa, 2012). Since the present model omits such transfer effects of synesthetic colors from other sets of graphemes, any already-acquired mappings between Hiragana characters and synesthetic colors might have disrupted the expected patterns of results in English grapheme-color associations. However, we do not think these are the only reasons for the low explanatory power of the model for the English alphabet letters in this study. Watson et al. (2012) conducted multiple regression analyses to predict hue distance with 11 predictors including shape similarity/difference (including Gibson's and Boles and Clifford's measures), letter frequency difference, ordinality difference, phonological (letter name) similarity. He found that even with native English speakers and even using as many as 11 predictors, the 11 predictors explained at most about 32% of the variance (details of this analysis were not supplied). Given that the participants in the study of Watson et al. (2012) were highly familiar with English alphabet letters, the reduced explanatory power of these regression models cannot be attributed to letter familiarity.

We propose a different explanation for the generally low explanatory powers of the second-order measures with the English alphabet. It hypothesizes that a synesthetic color highlights the most discriminating feature of each grapheme, which people (both synesthetic and non-synesthetic) rely on when they learn graphemes. In turn, this "marking" potential facilitates people's discrimination of a grapheme which then leads to their speedier learning of that grapheme. In doing so, we assume that not all properties of a given grapheme afford this "marking" potential nor do these markers apply to all graphemes. For example, ordinal numbers may provide effective discriminating cues for certain graphemes in a sequence i.e., from the first to the fifth, at most. However, it should be difficult to associate the larger ordinal numbers with successive graphemes. Other than the cases of the graphemes above (i.e., from the first to the fifth graphemes), information provided by the domain of ordinality may be rough, like either "early" or "later" in the sequence. This means that ordinality can afford effective "marking" potential only in some graphemes (e.g., the first to the fifth graphemes in a grapheme sequence, and graphemes that appear distinctively early or late in a sequence); furthermore, it should be difficult to distinguish all graphemes with assistance of only ordinality information. Similarly, it is likely that each of phonology, visual shape, and familiarity can afford effective "marking" potential only in some (different) graphemes. This reasoning leads us to hypothesize that multiple basic grapheme properties need to be utilized simultaneously to attach a "marker" to each of all graphemes.

We presume that certain basic grapheme properties such as ordinality, phonology, visual shape are preferentially used (when available) as "markers" in grapheme discrimination. The rationale of this is that they should be highly stable in the input to children because such properties are socially shared. If some of the basic grapheme properties are not useful in discriminating and learning graphemes (as in the case of phonology in discriminating English alphabet letters), children need to find other sources of information available for grapheme discrimination. Such additional sources might involve one's memory for properties associated with graphemes (e.g., colors of refrigerator magnets), word meanings (e.g., " 'B' is for 'banana' so the color of 'b' is yellow"), and so on. The additional sources will usually differ across individuals because they depend on personal experiences.

Note that the basic properties are relational, while the additional sources may usually have non-relational properties. We argue that this is the reason for the generally low explanatory powers of the second-order measures with the English alphabet in the present study and in Watson et al. (2012). Fewer relational basic grapheme properties are available in English alphabet letter discrimination due to the fact that phonology does not provide effective discrimination cues. This suggests that synesthetic colors for English alphabet letters may depend more on non-relational properties such as memories associated with certain graphemes (e.g., Witthoft and Winawer, 2006, 2013) and word meanings (e.g., Rich et al., 2005; Simner, 2007) than do these colors for Japanese Hiragana characters. Since the analyses in this study and in Watson et al. (2012) were designed to examine second-order relationships between relational grapheme properties and synesthetic colors, we conjecture that the regression models in these studies had only limited powers for explaining grapheme-color associations in synesthesia.

#### **CONCLUDING REMARKS**

We have proposed a new model that provides a comprehensive explanation of synesthetic grapheme-color association determination processes, including the differences across writing systems. It is based upon previous studies on developmental perspective of grapheme-color synesthesia (Rich et al., 2005; Asano and Yokosawa, 2012; Watson et al., 2012). Results of an accompanying experiment on synesthetic colors for Hiragana characters and English alphabet letters were generally consistent with the model. Together model and results suggest that the ways in which graphemes are introduced to children during development and are processed in the brain are critical. Indeed, these factors are more important in determining grapheme-color associations than grapheme properties *per se*. This challenges a view that synesthesia requires unusual hard-wired cross-associations between certain brain areas (e.g., Brang et al., 2010, 2011). The present model is partly consistent that of Brang et al. (2011) in assuming that some aspects of synesthetic colors are associated with basic visual components of letters in some ways. However, it differs in supporting the view that grapheme-color synesthesia builds on normal cognitive (including language) processing mechanisms (Simner et al., 2005; Simner, 2007; Asano and Yokosawa, 2011, 2012; Watson et al., 2012)).

The model of the present study assumes that linkages between color representations and representations of features in some domains (e.g., ordinality in conventional sequences, sounds, and visual shapes) are formed in the brain before grapheme acquisition. We believe this is possible based on the observations reported in several previous studies (e.g., Rich et al., 2005; Wagner and Dobkins, 2011; Asano and Yokosawa, 2012; see Introduction), however, direct evidence for this must await further studies.

According to the model proposed in this study, synesthetic colors function as "discriminating markers" of graphemes during learning graphemes, which are arbitrary symbols that may not be easily acquired by children. Certain questions are left unanswered: why colors are used as "markers"? How this is achieved in the brain? If the "discrimination markers" view holds true, why are colors also associated with other cognitive entities that people do not have to "acquire" (meaning that no special training is required to process them), such as music sounds and tastes, in other types of synesthesia, such as colored hearing and gustatorycolor synesthesia? Are these types of synesthesia qualitatively different from grapheme-color synesthesia? Future studies are required.

#### **AUTHOR CONTRIBUTIONS**

Michiko Asano and Kazuhiko Yokosawa designed the study and collected data. Michiko Asano analyzed the data and wrote the paper. Kazuhiko Yokosawa reviewed and refined the paper. Both authors discussed the results and commented on the manuscript.

#### **ACKNOWLEDGMENTS**

This research was supported by KAKENHI [Grant-in-Aid for Scientific Research (A) #22330199] awarded to Kazuhiko Yokosawa and Grant-in-Aid for JSPS Research Fellows (#23-2872) awarded to Michiko Asano, both from Japan Society for the Promotion of Science (JSPS).

#### **SUPPLEMENTARY MATERIAL**

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fnhum. 2013.00757/abstract

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 26 July 2013; accepted: 22 October 2013; published online: 11 November 2013.*

*Citation: Asano M and Yokosawa K (2013) Grapheme learning and grapheme-color synesthesia: toward a comprehensive model of grapheme-color association. Front. Hum. Neurosci. 7:757. doi: 10.3389/fnhum.2013.00757*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2013 Asano and Yokosawa. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Semantic mechanisms may be responsible for developing synesthesia

## **Aleksandra Mroczko-Wa**˛**sowicz <sup>1</sup>\* and Danko Nikoli´c 2,3,4,5**

1 Institute of Philosophy of Mind and Cognition, National Yang-Ming University, Taipei, Taiwan

<sup>2</sup> Ernst Strüngmann Institute (ESI) for Neuroscience in Cooperation with Max Planck Society, Frankfurt, Germany

<sup>3</sup> Frankfurt Institute for Advanced Studies, Frankfurt, Germany

<sup>4</sup> Department of Neurophysiology, Max Planck Institute for Brain Research, Frankfurt, Germany

<sup>5</sup> Department of Psychology, Faculty of Humanities and Social Sciences, University of Zagreb, Zagreb, Croatia

#### **Edited by:**

Beat Meier, University of Bern, Switzerland

#### **Reviewed by:**

Karsten Specht, University of Bergen, Norway Berit Brogaard, University of Missouri, USA

#### **\*Correspondence:**

Aleksandra Mroczko-Wa˛sowicz, Institute of Philosophy of Mind and Cognition, National Yang-Ming University, 155, Sec. 2, LiNong St., Beitou, Taipei 11221, Taiwan e-mail: mroczko-wasowicz@ hotmail.com

Currently, little is known about how synesthesia develops and which aspects of synesthesia can be acquired through a learning process. We review the increasing evidence for the role of semantic representations in the induction of synesthesia, and argue for the thesis that synesthetic abilities are developed and modified by semantic mechanisms. That is, in certain people semantic mechanisms associate concepts with perception-like experiences—and this association occurs in an extraordinary way. This phenomenon can be referred to as "higher" synesthesia or ideasthesia. The present analysis suggests that synesthesia develops during childhood and is being enriched further throughout the synesthetes' lifetime; for example, the already existing concurrents may be adopted by novel inducers or new concurrents may be formed. For a deeper understanding of the origin and nature of synesthesia we propose to focus future research on two aspects: (i) the similarities between synesthesia and ordinary phenomenal experiences based on concepts; and (ii) the tight entanglement of perception, cognition and the conceptualization of the world. Importantly, an explanation of how biological systems get to generate experiences, synesthetic or not, may have to involve an explanation of how semantic networks are formed in general and what their role is in the ability to be aware of the surrounding world.

**Keywords: synesthesia, perception, cognition, concepts, learning, semantics, cognitive penetrability**

## **INTRODUCTION**

Following a long history of interdisciplinary research on synesthesia (see e.g., Locke, 1689; Galton, 1883/1907/1973; Mahling, 1926; Peacock, 1988; Cytowic, 1995, 2002; van Campen, 1999; Day, 2005; Dixon and Smilek, 2005; Jewanski and Sidler, 2006; Macpherson, 2007; Brogaard, 2013; Auvray and Deroy, 2014), at the beginning of the 21st century an important debate has emerged about the fundamental nature of synesthesia. Initially, the discussion was dominated by the explanation of synesthesia in terms of intermixed senses. It was presumed that a sensory input in one modality directly elicits an additional sensory activation in another sensory modality, and this was further deduced to be based on direct connections between the respective brain areas, known also as the cross-activation hypothesis (e.g., Baron-Cohen and Harrison, 1997; Ramachandran and Hubbard, 2001a; Cytowic, 2002).

However, meanwhile, this interpretation has encountered problems. It has become apparent that in many forms of synesthetic associations, only the concurrents exhibit clear sensorylike properties, while the inducers take the form of concepts. For example, one of the most common forms of synesthesia is time-unit synesthesia (Smilek et al., 2007a; Jarick et al., 2008, 2009, 2011; Brang et al., 2013b) in which weekdays or months are associated with colors, distinct spatial positions and with other concurrents. Importantly, there are no direct sensory inputs for time units. One cannot directly see June or Thursday in a way one can see a car or letter "A". Time units exclusively exist as concepts. But time-unit synesthesia is not the only case. For other forms of synesthesia the traditional, sensory-to-sensory characterization has also been found unsatisfactory (Ward and Simner, 2003; Ward, 2004; Simner, 2007; Nikoli´c et al., 2011; Mroczko-Wa˛sowicz and Werning, 2012). Thus, it has been repeatedly suggested that a full account of the phenomenon should go beyond the standard sensory-sensory approach (e.g., Dixon et al., 2000, 2006; Simner and Ward, 2006; Ward et al., 2006; Ward and Sagiv, 2007; Sagiv et al., 2011; Eagleman, 2012; Jürgens and Nikoli´c, 2012, 2014; Simner, 2012; Watson et al., 2012a, 2014; Mroczko-Wa˛sowicz and Nikoli´c, 2013; Chiou and Rich, 2014; Brogaard, 2014; Brogaard et al., 2014).

A number of different remedies to the problem of an adequate characterization of synesthesia have been proposed. An attempt to bridge this discrepancy between the theory and the data was to postulate individual differences among synesthetes (Dixon and Smilek, 2005; Hubbard et al., 2005; Cytowic and Eagleman, 2009; Rouw and Scholte, 2010; Rouw, 2011; Simner, 2013). Thus, there would be two main groups, higher and lower synesthetes, depending on whether their synesthesia is triggered by the conceptual or by the sensory properties of a stimulus, respectively (Ramachandran and Hubbard, 2001b). Another approach was to suggest that the phenomenon oscillates between these two types of inducers within one and the same individual (Hubbard et al., 2005; Ward et al., 2007).

It can be said that the last decade experienced a renewed interest in re-conceptualizing and re-defining the phenomenon of synesthesia (Simner, 2007, 2012; Sagiv et al., 2011; Cohen Kadosh and Terhune, 2012; Mroczko-Wa˛sowicz and Werning, 2012; Jürgens and Nikoli´c, 2014). Researchers have started to question the exclusively sensory interpretation of synesthesia and have suggested that the phenomenon also involved high-level cognitive representations (Ward et al., 2006, 2007; Ward and Sagiv, 2007). This includes the proposal that synesthesia is exclusively a semantically induced phenomenon (Nikoli´c, 2009; Chiou and Rich, 2014). This suggestion severs the traditional distinction between perception and cognition assumed for a long time in philosophy, psychology and cognitive science (Mroczko-Wa˛sowicz and Werning, 2012; Mroczko-Wa˛sowicz, 2013). Evidence for this alternative, extended view of synesthesia is continually increasing (Dixon et al., 2000, 2006; Grossenbacher and Lovelace, 2001; Elias et al., 2003; Mroczko et al., 2009; Nikoli´c et al., 2011; Simner, 2012; Rothen et al., 2013; Jürgens and Nikoli´c, 2014). The aim of the present paper is to examine these fundamental issues and the particular forms of the phenomenon for which the hypothesis applies that semantic mechanisms create synesthesia.

## **MIND-DRIVEN HIGHER SYNESTHESIA**

In synesthesia, an attended and recognized stimulus leads to additional phenomenal experiences, that is, to experiential states that normally do not occur (Rich and Mattingley, 2003; Mattingley, 2009; Nijboer and Van der Stigchel, 2009; Mroczko-Wa˛sowicz and Nikoli´c, 2013). As a result, the stimuli corresponding to the inducer and the experiences associated with the concurrent form a highly integrated percept—a phenomenally unified experience (Mroczko-Wa˛sowicz and Werning, 2012; Mroczko-Wa˛sowicz, 2013). This unified experience may cover the same sensory modality, or different sensory modalities, or even different domains e.g., cognitive, sensory and motor. The resulting conscious experiences are typically unified, meaning that all the experiences at a given time are present simultaneously forming an overall phenomenal perspective—a single encompassing phenomenal state. The different component experiences are said to be co-conscious parts or aspects of the subsuming state that has a conjoint phenomenology, or a joint phenomenal content (Shoemaker, 1996, 2003; Bayne and Chalmers, 2003; Tye, 2003; Brook and Raymont, 2013). As such, they are intimately linked and integrated from the perspective of the self. Consequently, a person's perceptual and cognitive states result in a synchronic phenomenal unity (Raymont and Brook, 2009; Bayne, 2010; Mroczko-Wa˛sowicz, 2013).

One of the most important issues in the current debate about the phenomenon of synesthesia pertains to the putative causal mechanisms underlying synesthetic associations. The question of whether synesthesia is a lower or higher phenomenon translates into a question of whether novel synesthetic associations can be explained by direct synaptic connections between neurons representing respectively the inducer and the concurrent—which is a possible explanation of the low-level hypothesis. Alternatively, synesthetic associations may occur through a high-level of system organization, that enables associations to be formed in a more elaborated, distributed and flexible manner than what can be accounted for by direct synaptic connections between sensory areas. The latter would correspond to the high-level hypothesis of synesthesia and is closer to the level at which the semantic structure of knowledge operates: rich association systems involving additional brain areas as opposed to direct connections between two sensory areas of the brain (for a recent hypothesis on the involvement of ventral IT-cortex see Chiou and Rich, 2014). At this higher level a prominent role is played by context, attentional mechanisms, and interpretation of the stimuli.

The two levels are often referred to as *perceptual* and *conceptual* (or *cognitive*), respectively (Ward and Simner, 2003; Eagleman, 2012; Simner, 2012; see also Evans, 1982). This is because it is typically assumed that perception is an earlier and hence, more rudimentary stage of information processing than is the activation and processing of semantic information. Nevertheless, the distinction between higher and lower synesthetes responding to respectively, high- and low-level features of inducing stimulus has rarely been a topic of direct investigation.

Many forms of synesthesia indicate the involvement of semantic processes i.e., of the meaning of the stimulus. Synesthetes often associate the same concurrent (e.g., the same hue of a color) with different physical representations of the stimulus, i.e., with different sensory instantiations of the same concept (e.g., the Arabic digit "4", the Roman numeral "IV", the word "four", and four dots on a dice; Ramachandran and Hubbard, 2001b; Ward et al., 2007). Number-color synesthesia can be induced via sensory presentation (e.g., digits), but also conceptually via cardinality (the quantity) or via another conceptual property of ordinality (the position in a sequence).

There are two other common forms of synesthesia that can be mind-driven and that are variants of the already mentioned timeunit synesthesia: the first is known as *colored sequence synesthesia* in which names of time units, such as days of the week and months of the year, are colored differently from their graphemic constituents (Simner et al., 2006; Tomson et al., 2013), and the second is known as *spatial sequence synesthesia* in which ordinal categories involving numbers, time and alphabet elicit experience of spatial forms (Sagiv et al., 2006; Eagleman, 2009).

A related phenomenon observed in non-synesthetes, which suggests a bridge between synesthesia and non-synesthetic perception, is the spatial numerical association of response codes. Here, overlearned ordinal stimuli are associated with implicit spatial representations (Dehaene et al., 1993).

Some neuroanatomical studies have suggested that differences between higher and lower synesthetes map onto another, more empirically grounded, distinction between associators and projectors (Ramachandran and Hubbard, 2001a,b; Dixon et al., 2004; Dixon and Smilek, 2005; Hubbard et al., 2005; Rouw and Scholte, 2010; van Leeuwen et al., 2011). The assumption is that a semantic representation of number (relevant for higher synesthetes) is likely linked to a more conceptual representation of color (relevant for associators experiencing their colors internally in their "mind's eye"). However, recent evidence seems to challenge this view suggesting that the two distinctions are not the same but rather orthogonal (Ward et al., 2007). Ward and colleagues propose that the mechanisms that give rise to the associator-projector distinction are independent of those that give rise to higher-lower characteristics. According to the researchers, the former reflects an internal vs. external frame of spatial reference related to synesthetic phenomenology, and the latter reflects the extent to which conceptual vs. sensory properties of the stimulus are involved in the induction of synesthesia. Along with the results of the behavioral tests and subjects' phenomenological reports, they found that higher synesthetes were not necessarily grapheme-color associators, but could also be projectors.

As an explanation of the finding Ward et al. (2007) proposed to differentiate between surface-projectors and near spaceprojectors. Surface projectors evoke an externalized frame of reference relative to the location of the stimulus, perceiving their synesthetic color concurrents as located in the same spatial location as the inducing graphemes, dice patterns, finger counting, etc. In contrast, near space-projectors elicit an externalized frame of reference relative to the location of their body. For example, the concurrent may always be located 20 cm in front of the perceiver. This dissociation may help explain varying results for projectors. Projectors are shown to be faster in naming their synesthetic colors than veridical colors, and associators exhibit the opposite effect (Smilek et al., 2001, 2003). However, projectors who project colors into external space but not onto the surface of graphemes may behave more like associators on the task of naming their concurrents. Near space-projectors may be slower with synesthetic colors than with ink colors as a result of the need to shift their attention from the grapheme location to the color location. In contrast, surface-projectors place concurrents at the same location as the attended stimuli, which may then enable more efficient naming of synesthetic colors.

The Ward et al. (2007) study also suggests that it may well be possible for a synesthete to demonstrate higher synesthesia in one respect but lower in another, e.g., experiencing number forms may be shared by two groups, associators and projectors. The respective internal and external location of spatial forms does not have to reflect the spatial reference of their experiences when, for example, reading graphemes. One and the same synesthete may be an associator in his number forms but a projector for his grapheme-colors. In line with this, a comparable oscillation between high and low synesthesia may occur within the same person for the different forms of synesthetic associations the person possesses.

This view is supported by studies on numerical cognition (Cohen Kadosh et al., 2005; Cohen Kadosh and Henik, 2006a,b) in which digit-color projectors showed a characteristic feature of higher synesthesia when linking colors with the meaning of numbers or with information on magnitude. Similarly, both projectors and associators showed Stroop-like effects when perceiving color for an arithmetic sum in which the result was implied and not visually presented and hence, had to be extracted through semantic mechanisms (Dixon et al., 2000; Jansari et al., 2006). For lower synesthetes, color would have to be linked to the visually presented physical form of the digit, and not to the conceptual aspects of a number. Hence, it is possible that for different individuals the semantic structures that lead to synesthesia are organized differently. Some synesthetes may be higher in the sense of visuo-spatial properties such as the number forms and ordinality (and spatial forms for time and the alphabet since these concepts also possess ordinality), while other synesthetes may be higher in the sense of numerosity (cardinal aspects of number meaning, number concepts). And yet, there could be a group that combines the two effects, which then results in having the same color for "January" and "1", "February" and "2", etc. (Sagiv et al., 2006).

Examples of cases of synesthetic experiences arising in the absence of any direct impact or physical presence of the stimulus, or occurring independently of the various physical forms that external inducers may take, are mind-dependent or mind-driven higher synesthesia, also known as ideasthesia. We suggest that these experiences are driven by semantic mechanisms as a part of mental representations whereby each synesthete's individual semantic network contains concurrents as a part of the meaning of the inducing stimuli (Jürgens and Nikoli´c, 2014). A similar proposal for a conceptual contribution to synesthesia was made by Meier (2013). Meier's approach is based on the research on implicit bidirectionality (Brugger et al., 2004; Johnson et al., 2007; Weiss et al., 2009), and states that processing of an inducer is affected by a spreading of implicit activation of a concurrent. This is in agreement with our claim that synesthetic associations include unconventionally coupled supplementary phenomenal features as a part of the semantic knowledge network. Therefore, synesthetic experiences do not have to rely on external stimuli to be induced, i.e., cognitive or concept-dependent mental states are reported to be sufficient to elicit synesthetic perceptual concurrents in the absence of any related physical stimulation (Nikoli´c et al., 2011; Mroczko-Wa˛sowicz and Werning, 2012).

The above-mentioned case of a digit-color synesthete studied by Dixon et al. (2000) using mental arithmetic, provided a striking example of conceptual synesthesia. Not only viewing digits triggered a visual experience of a specific color. Also a mental activation of the concept of the calculated number was fully sufficient to evoke synesthesia (e.g., even without writing down the result of the calculation) (**Figure 1**). This was revealed when the synesthete performed mathematical additions such as "4 + 3" followed by a patch colored congruently or incongruently to the synesthetic color associated with the result of the summation (e.g., yellow for digit 7). The result of summation was never directly presented. The synesthete had to name the color

**FIGURE 1 | An example of the stimuli used in the experiment of Dixon et al., 2000)**.

of the patch before reporting the result of the summation. The responses were slower in the case of incongruency i.e., when the color of the patch was different from the color associated with the arithmetical sum. This Stroop-like test objectified the minddriven case of synesthesia demonstrating that the cause of this synesthetic experience is not provided by the immediate sensory stimulation.

Another source of evidence of mind-dependent synesthesia has been provided in studies with letter-color synesthetes reporting to experience the same concurrent to different physical stimuli representing the same *type*, *kind* or *semantic category*. Experiences were the same for letters presented in different alphabets (Mroczko et al., 2009), various fonts, or presented via different sensory modalities—visual or auditory (Simner, 2007). Similarly, number-color synesthetes show indifference to a wide range of physical instantiations of numerosity (Dixon et al., 2000; Ward and Sagiv, 2007).

Such sets of low-level sensory features or objects identified as members of a familiar meaningful category are the subject of recent philosophical discussions about object recognition and the related high-level vs. low-level content of perception referring to a *kind* property (i.e., when recognizing that something belongs to a certain kind or type, like being a tiger, being a pine tree, etc.) (Pylyshyn, 1999; Siegel, 2006; Bayne, 2009). The same analysis might apply to typical synesthetic inducers such as seeing letters "A", "A" and "a" all *as "*A", that is, recognizing an exemplar of a certain semantic category (i.e., a token of a type). There is no consensus on how to interpret such content, as high-level conceptual or low-level perceptual, but an agreement is that this content is cognitive, at least in some general sense.

A related issue is that of the mode of recognition that applies to such a kind property (Simner, 2007; Auvray and Deroy, 2014; see Pylyshyn, 1999 for the argument that a computation based on perceptual principles inherent to categorical perception may lead to low-level detection of similarities between stimuli). We propose that at least for some forms of developmental synesthesia, if not for all, the concurrent is activated only after semantic decoding of the inducer is completed—i.e., the meaning of inducer must be extracted before concurrent can be experienced. As mentioned, a number of studies provide convincing evidence for the role of conceptual contents in the induction of synesthesia (see **Figure 2** for an illustration of that proposal).

Thus, the studies confirm that synesthetic experiences are largely internally-driven by mental representations of objects, higher-order types and semantic categories, cognitive states and events, or other internal determinants such as concepts, thoughts, moods, memories and imagery (Spiller and Jansari, 2008; Rothen et al., 2012; Meier and Rothen, 2013; Simner, 2013; Ward et al., 2013). Another form of higher synesthesia has been observed in two different individuals who acquired their color/shape synesthesia for digits and mathematical formulas; interestingly the visual concurrents were generated in higher parietal and frontal brain regions (Bor et al., 2007; Brogaard et al., 2013). Since the cognitive representations seem to be factors sufficient to elicit synesthesia they may also play a role in explaining variations of synesthetic experiences.

In the following section, we make the point that conceptual properties are shared across many forms of synesthesia. This makes us consider higher (semantic) synesthesia a separate class, which ought to have its own place in the synesthetic taxonomy and has to be considered accordingly in the definitional and explanatory efforts of investigations into the phenomenon.

## **CONCEPTUAL ASPECTS AND SEMANTIC MECHANISMS INVOLVED IN DIFFERENT FORMS OF SYNESTHESIA**

To substantiate the importance of studying higher synesthesia, we highlight numerous conceptual aspects and semantic mechanisms involved in various types of synesthesia. An account directly addressing the influence of semantic information on the induction of synesthetic experience is "ideasthesia" which literally means *sensing concepts*, and refers to the conceptual processing underlying synesthesia in which only concurrents are perceptual while inducers are conceptual (Nikoli´c, 2009). According to this model, high-level semantic mechanisms assign low-level sensory concurrents (e.g., synesthetic colors are triggered only when a synesthete extracts the meaning of the presented grapheme; see also Jürgens and Nikoli´c, 2012, 2014; Milán et al., 2013; cf. Brang et al., 2013a).

Synesthetic associations are not just cross-modal, but they also cross domains, i.e., besides the various modalities of senses, these associations may involve domains of bodily, motor and emotional states as well as domains of abstract, conceptually-represented entities like numbers or time units. Cases of synesthetic inducers going beyond traditionally denoted sensory modalities have been found for activities such as reading musical notes, calculating, imagining, or just thinking of a stimulus (Frith and Paulesu, 1997; Grossenbacher, 1997; Dixon et al., 2000, 2006; Ramachandran and Hubbard, 2001b; Cytowic, 2002; Rich et al., 2005; Ward et al., 2006; Spiller and Jansari, 2008; Nikoli´c et al., 2011).

Studies with bilingual grapheme-color synesthetes have shown that synesthetesia can be experienced also for alphabets different from the one used in the first language (Mills et al., 2002; Rich et al., 2005; Witthoft and Winawer, 2006). In such a crosslinguistic transfer synesthetic colors from a second language are usually mapped onto the existing colors of the first language (Simner, 2007; Mroczko-Wa˛sowicz and Nikoli´c, 2013). Often, the color is associated with a letter by its similarity to other letters, i.e., letters of similar shapes tend to elicit similar colors (Brang et al., 2011a; Jürgens and Nikoli´c, 2012, 2014).

Results from our own studies suggest that synesthetic associations to new graphemes are established quickly, and are not created from scratch but are inherited from existing associations, the concurrent colors being passively adopted from the original synesthesia (Mroczko et al., 2009; Mroczko-Wa˛sowicz and Nikoli´c, 2013). To demonstrate this, we conducted a study with 16 grapheme-color synesthetes in which we attempted to replicate in the laboratory conditions the natural phenomenon of crosslinguistic transfer. We introduced Glagolitic graphemes (novel shapes, without any corresponding sound), which come from an ancient Eastern European writing system that was ideal for our study; it had the sufficient exotic appearance. Only few letters

bore any physical resemblance to other known graphemes (see **Figure 3**; Franoli´c and Žagar, 2008).

We assigned to each new grapheme a meaning by linking it to existing Latin letters and Arabic numerals. A simple writing exercise with paper and pencil sufficed to imitate the natural crosslinguistic transfer. The exercise lasted less than 10 min during which subjects learned how to write a Glagolitic grapheme and then used this knowledge to write 20 words or number sequences. The subjects substituted one familiar Latin/Arabic grapheme with its Glagolitic counterpart (as in **Figure 3B**). This resulted in a quick transfer of the original synesthetic color experiences from their native Latin alphabet to Glagolitic graphemes never seen before the experiment (Mroczko et al., 2009). Also, Stroop-type tests (Stroop, 1935; Mills et al., 1999; Nikoli´c et al., 2007) indicated that these new synesthetic associations were immediate and involuntary.

The novel synesthetic associations were created within minutes, which was too rapid a process to be accounted for by low-level sensory cross-wiring between grapheme and color brain areas. Low-level perceptual learning is usually not cognitively penetrable (see below) and, as a result, it is a relatively slow process requiring typically thousands of trials (Ahissar and Hochstein, 1997; Goldstone, 1998; Seitz and Watanabe, 2005). In contrast, the meaning of a new symbol can be learned quickly (often in a single trial; Markson and Bloom, 1997; Bloom, 2000). Thus, shorter transfer times of synesthetic associations are expected if the associations originate at the semantic level of representation. In other words, the speed of transfer is indicative of the type of representation (high- vs. low-level).

Additionally, the new associations generalized to the exemplars of different physical representation (handwritten by another person), which is consistent with learning a semantic category of stimuli rather than individual exemplars, or particular motor coordinates, and is also inconsistent with perceptual learning. The ways in which these stimuli were similar can be described only as semantic "similarities", i.e., they shared the same meaning and not elementary visual features presumably processed in the grapheme area (Brang et al., 2011a).

Therefore, a better explanation of grapheme-color synesthesia can be provided by high-level conceptual mechanisms than by low-level associations: arbitrary synesthetic colors can be assigned to novel graphemes through transfer of semantic knowledge. This suggests the involvement of the high-level stimulus properties in eliciting synesthetic colors. The newly created synesthetic associations to Glagolitic letters must have been induced via existing synesthetic associations previously linked to conceptual representations of the Latin letters. In other words, after training, a Glagolitic letter would activate the same semantic content that is normally activated by a Latin letter. Here, as for the other cases of higher synesthesia, the meaning of the stimulus has to be extracted before the concurrent phenomenal aspect can be included into an overall unified synesthetic experience (Edquist et al., 2006; Simner, 2007; Mroczko-Wa˛sowicz, 2013). Therefore, synesthesia seems to rely essentially on a certain conceptual representation of the stimulus.

The significance of meaning and context in grapheme-color synesthesia was demonstrated also in the study "Is the sky 2?", using electroencephalography to measure event-related potentials (Brang et al., 2008). The authors concluded that connections between colors and numbers are bidirectional and that concurrent color sensations are treated by synesthetes as meaningful stimuli. The impact of semantic activation and linguistic modulation such as numerical magnitude or the frequency of grapheme use in a language may also be reflected in the saturation and luminance of the experienced colors (Beeli et al., 2007; Cohen Kadosh et al., 2007; Smilek et al., 2007b; Watson et al., 2012b; see also work by Akins et al. on different languages' effects upon grapheme-color synesthesia). Moreover, synesthetic colors elicited by ambiguous graphemes are modified by context. They depend on the interpretation of the grapheme either as a letter or as a digit (e.g., the grapheme "V" referring to Latin letter and Roman numeral; Ramachandran and Hubbard, 2001b; Dixon et al., 2006).

Another example providing support for the view that semantic mechanisms may be responsible for developing synesthesia, is number-color synesthesia. Here, it is the higher cognitive numerical representation, the conceptual representation of a number (numerosity, quantity), that is responsible for eliciting synesthetic concurrent experiences (Butterworth, 1999; Cohen Kadosh et al., 2005; Cohen Kadosh and Henik, 2006a,b). In such a case, same colors are elicited from physically different representations of a number (e.g., Arabic and Roman numerals, arrays of dots, dice patterns, finger counting or number names), provided they match in the indicated quantity (Dixon et al., 2006; Ward and Sagiv, 2007; Ward et al., 2007).

In case of projectors, colors may be projected not only on the surface of a page when viewing written digits, but also on the surface of a dice or on synesthete's fingers when counting. The color reported for all these different physical instantiations of number concept is the same. The cardinality is the direct cause of the sameness of color across various notational formats (Ward and Sagiv, 2007). During development, the synesthetic cardinality-color associations may become generalized to other education-based and culturally acquired symbols. Other forms of synesthesia, described above, confirm that color concurrents may transfer from one physical system of representation to another on the basis of semantic correspondences, due to shared conceptual properties (Mroczko et al., 2009; Jürgens and Nikoli´c, 2012). For some synesthetes the colors of the writing system from a second language are taken from the colors of the alphabet of their first language (Mills et al., 2002; Witthoft and Winawer, 2006; Mroczko et al., 2009; Mroczko-Wa˛sowicz and Nikoli´c, 2013), or the synesthetic color experience associated with a written musical note acquires the color of the name denoting it (Ward et al., 2006).

Other examples might be number-forms synesthesia, timeunits or calendar synesthesia, all of which involve experiencing numbers or time-units within a space. Thinking about sequential concepts, such as numerical sequences in a form of days of the week or months, elicits spatial forms and visualized mental maps that contain arranged numbers (Sagiv et al., 2006). For example, a subject may experience units of time as being arranged in an ellipse, column or a spiral. Moreover, this spatial structure is usually placed at a specific spatial location within the three-dimensional space surrounding the body of the synesthete. Alternatively, the structure is located within a virtual space not positioned relative to the body but within the mind's eye (e.g., Smilek et al., 2007a; Jarick et al., 2008; Mann et al., 2009; Simner et al., 2009).

Days of the week as well as months present maybe the most apparent form of a conceptual inducer, as they do not possess any characteristic sensory properties. There is nothing perceptually specific that distinguishes a Monday from a Wednesday or May from August.

Another illustrative example of how concepts have impact on synesthetic experiences is given by lexical-gustatory synesthesia. In this case, different flavors are experienced in response to various words. Interestingly, such synesthetes can perceive a gustatory sensation even before articulating a given word thus, while the word is still on the tip of their tongue i.e., in the semantic phase (Simner and Ward, 2006). This demonstrates that the physical instantiation of the stimulus (i.e., the somato-sensory experience of verbalization, the spoken expression of a thought in words) is not necessary for eliciting synesthetic sensations of a flavor. The activation of the concept, which is grasped mentally, is fully sufficient.

Higher cognitive aspects of inducing synesthesia can furthermore be identified in mirror-touch or mirror-pain synesthesia. When seeing another person being touched or being in pain, any person may trigger corresponding emotions and empathy. In synesthetes however, in addition, a respective tactile or painful experience may be produced. And this experience has the quality of real touch and pain, even being located on the equivalent part of synesthete's body (Blakemore et al., 2005; Banissy and Ward, 2007, 2013; Fitzgibbon et al., 2010).

To the list of synesthesias shown to be based on semantic interpretation of the stimulus belongs also a recently discovered swimming-style-to-color-synesthesia in which seeing another person swimming with a given swimming style, or even just thinking about a certain swimming style, elicits synesthetic color experience (Nikoli´c et al., 2011; Mroczko-Wa˛sowicz and Werning, 2012; Rothen et al., 2013) (see **Figure 4**).

Collectively, all these findings impose constraints on theories of the neurophysiological origins of synesthesia. Synesthetic experiences often involve inducers that are not strictly sensory

(i.e., words, letters, numbers, time units, musical notes, personalities, emotions, or movement styles). Conceptual components, identified in many forms of synesthesia, transcend the traditionally believed sensory nature of inducers. Synesthesia seems to rely on a certain interpretation of the stimulus and the meaning that it has for the subject, and this meaning may change dynamically depending on the context (Rich and Mattingley, 2003; Dixon et al., 2006; Ward et al., 2006).

## **COGNITIVE PENETRABILITY AND SYNESTHESIA**

These conclusions suggest that some forms of synesthesia may be understood as cases of cognitive penetrability under the conditions in which the sensory experience normally is not penetrable. Pylyshyn (1999) introduced the concept of cognitive penetrability and the lack thereof, to account for the fact that in some cases visual experience of a perceived object is fully independent of how we cognitively understand the object or what we know about that object. For example, in the Mueller-Lyer illusion one of the two lines continues to appear shorter irrespective of the direction of attention, our knowledge that the lines are equally long, or of any other form of mental effort. Thus, in this case the perceptual experience cannot, in any known way, be affected by cognition. In other words, the sensory property of visual experience is cognitively impenetrable. Pylyshyn referred to these impenetrable aspects of vision as early vision.

Synesthesia may be a case in which cognitive penetration also takes place for early sensory processing. For example, the perception of colors is usually considered impenetrable, i.e., cognitive operations cannot change the experience of the color of a perceived object (Brogaard and Gatzia, in press; but cf. Macpherson, 2012; Siegel, 2012; Vetter and Newen, 2014). At least this seems to be the case for non-synesthetes. Synesthetes are special in that they are able to penetrate this early aspect of vision (or other sensory modalities) by certain higher cognitive mechanisms. This means, synesthetic perceptual concurrents can be modified by conceptual contents, contextual expectations, linguistic modulation and cultural factors being their inducers (Dixon et al., 2000; Simner, 2007, 2012; Mroczko et al., 2009; Meier, 2013; Mroczko-Wa˛sowicz and Nikoli´c, 2013; Brogaard et al., 2014). Activating a concept synesthetes can produce a topdown influence on a sensory experience. Thus, synesthetes exhibit a specific difference from non-synesthetes in the structure of their cognition. The ubiquity of this difference is relatively small, as it applies to a small subset of the concepts that a given person possesses and uses throughout the lifetime. For example, it may apply only to certain graphemes or words. Nevertheless, the very existence of such penetrability is important for the science of mental phenomena. It tells us that the organization of the mind is more flexible than usually presumed.

## **THE PHYSIOLOGY OF SEMANTIC REPRESENTATIONS**

Usually, one expects to assign to any mental event or faculty a brain region that is being considered its physiological underpinning. For example, if we consider associations between auditory inducers and visual concurrents, it is expected that there are distinct brain areas for each and that the physiological account solely involves those two brain areas. Similarly, if semantic representations are included in understanding the functioning of synesthesia, one automatically expects that a corresponding third brain area will be identified and will become a mediator of the flow of activation, i.e., from auditory, through semantic, to visual area.

In fact, this traditional one-area-for-function approach has already been applied to the semantic processes in synesthesia. Chiou and Rich (2014) propose that the anterior temporal lobe (ATL) is a hub for inducing synesthesia and that a transcranial magnetic stimulation (TMS) can lead to a breakdown of the coupling between inducers and concurrents. If their proposed research agenda is executed, we will probably soon know whether this part of the cortex plays a certain role in forming synesthetic associations.

However, there are reasons to believe that, irrespective of the results those future studies bring, this may not be the whole story. Semantics may be something that is not particularly localizable, but is instead to a high degree a distributed property of the brain. For example, there is evidence that the motor cortex plays an important role in the acquisition of action-related concepts (Gallese and Lakoff, 2005). Thus, the role of the motor cortex may be particularly important when a subject needs to prepare for a behavioral interaction with the object. Hence, the outcome of the question of whether semantics can be localized or whether it is a distributed function also depends on what is meant by semantics. In a generalized form, a concept may serve an extended function, not only to give a label to the stimulus, but also to prepare the person to interact with the conceptualized stimulus. If this interaction requires manual operations, the corresponding part of the motor cortex may be as important as the part of the infero-temporal cortex (Barsalou et al., 2003; Gallese and Lakoff, 2005).

Therefore, to fully account for the physiological underpinnings of the semantic mechanisms in synesthesia, we propose that future studies should not focus on a limited, pre-specified set of brain regions. Instead, a brain-wide analysis should be conducted with equal emphasis on all of its parts. In the past, such nonrestrictive analyses have been tremendously useful. For example, the analysis of connectivity in grapheme-color synesthesia (Rouw and Scholte, 2007) has shown that anatomically much more changes than connections between grapheme area and color areas. Anatomical differences associated with synesthesia are wide spread across the brain. We suggest that these anatomical changes reflect the widespread nature of semantics in the cortex. That is, semantics may be a function of the entire cortex, and not a function of a spatially bound module. In other words, if we accept the premise that synesthesia is largely a semantic phenomenon, the findings by Rouw and Scholte reveal the ensuing anatomical changes of a global network. This global system may be serving the various semantic aspects of graphemes and may play a role in its entirety when synesthetic concurrents are being assigned to those graphemes.

Meanwhile, the evidence has considerably expanded that synesthesia is not a function of local cortical circuits but a global phenomenon (Hänggi et al., 2011; Hupé et al., 2012), and so did the support for the arguments that synesthesia relates to high-level cognitive functions and abilities, including memory and creativity (Rothen and Meier, 2010; Rothen et al., 2012; Mulvenna, 2013; Meier, 2013). One implication is that the research on semantics in the brain is itself incomplete. Thus, synesthesia research does not have the luxury of building on well-established physiological underpinnings of semantics. Rather, synesthesia research has to coevolve with the research on semantics, helping us understand better semantics of natural language and cognition—a feat critically important for cognitive science. Therefore, in future studies, research on how our brain processes meanings may evolve handto-hand with research on synesthesia.

In this effort, the recently proposed theory of the organization of adaptive system, named practopoiesis, may provide valuable guidance (Nikoli´c, 2014). According to this theory, semantic mechanisms are closely related to the mechanisms of general knowledge stored in long-term memory on the basis of which more specific knowledge is extracted, which is stored in working memory. The theory suggests that not only phenomenal experiences in synesthesia, but also everyday experiences, may be related to the process of categorization—synesthesia being a useful trait to enhance this process (see also Jürgens and Nikoli´c, 2014). Synesthesia may thus develop in childhood to assist activation and manipulation of particularly abstract semantic contents.

These conclusions lead to a very different set of hypotheses that can be formulated about synesthesia than the traditional hypotheses on brain mechanisms known as cross-activation hypothesis (i.e., based on neuronal excitation) (Ramachandran and Hubbard, 2001a; Hubbard et al., 2005) and its alternative, disinhibition hypothesis (Armel and Ramachandran, 1999; Grossenbacher and Lovelace, 2001). We discuss those novel predictions next.

## **DEVELOPMENT OF SYNESTHESIA**

In case that synesthesia is a phenomenon the induction of which is predominantly semantic, at least for some forms of synesthesia, the development of synesthetic associations has to be related to the development of person's semantic knowledge base. The process of learning, education and acquiring knowledge about the surrounding world should in some instances give rise to synesthetic associations. Insights into how a child's learning is associated with synesthesia can be identified from the characteristics of the most common types of synesthesia. A pertinent question is the following: why are the letters, numbers and the time-units the most common inducers of synesthesia (Simner et al., 2006; Cytowic and Eagleman, 2009). We propose that these kinds of inducers are frequently the first truly abstract concepts that a child has to learn, at least in Western civilization.

The following set of hypotheses can be formulated: until a child begins to learn letters, numbers and days of the week, he or she is mostly faced with very concrete objects, such as particular toys, food, items of clothing, etc. These objects have very specific perceptual representations, which can be memorized and hence, are relatively easy to manipulate mentally. But then comes the time when a child must learn abstract concepts. One cannot see Tuesday or Sunday. There are no basic and straightforward perceptual memories to be formed for days of the week. Similarly, numbers (number names) are not just words but they have an abstract meaning of quantity that can apply to any objects (Teddy bears, bananas or shoes). This is one step more abstract than what the child was used to because none of the perceptually wellsupported images can be used as an identifier of numerosity (there could be three Teddy bears, three bananas and three shoes but nothing can explicitly visualize the meaning of number three). Likewise, one and the same letter can stand for many different words and thus, for many different objects.

As kids, we all have to find a way to acquire those abstract concepts. Synesthetes may have an advantage there. They can assign to those concepts something concrete, perceptual-like, that will always be present irrespective of the actual objects in the external world. They can assign a color to the concept, or a spatial position, or both. That way, they can accelerate their mental manipulations of those concepts: recall them quicker and more accurately, determine the relationships between them, and so on.

Synesthesia then becomes a tool for accelerating mental operations with cognitively demanding i.e., abstract, contents. This mostly aids the minds that have not yet been elaborated and have not developed the advanced, adult-like, abstract mechanisms of thinking. That is, this helps primarily the child's mind. And this explains why the most frequent forms of synesthesia involve letters, numbers and time units. Longitudinal studies of synesthesia starting in early childhood and reaching into the adult years will likely be of great help as a way to shed light on these processes (Simner and Bain, 2013; Meier et al., 2014).

## **DISCUSSION**

Generally, perceptual experiences are richer and more finegrained than our conceptual apparatus, i.e., they contain more distinct dimensions and perceptible positions on each dimension. Non-conceptual contents of perceptual states are more concrete and specific than are general conceptual contents of cognitive states (Evans, 1982; Heck, 2000; Peacocke, 2001a,b). Thus, our ability to discriminate perceptually particular sensory values exceeds the ability to conceptualize them. For example, we can notice the difference between specific hues of blue, such as Blue No. 37 and Blue No. 38, when presented simultaneously. Nevertheless those hues remain ineffable nuances to most of us (Raffman, 1995; Metzinger, 2009).

What we learn from studying the semantic aspects of synesthesia is that for synesthetes, unlike for non-synesthetes, even abstract concepts receive some of these fine-grained qualities to be specifically seen, heard, smelt, tasted, felt, or otherwise experienced. For example, digits have certain, very distinct, personalities, musical notes are precisely colored, words contain specific flavors or colors, time units may be positioned at particular locations in space, and emotions can similarly smell and sound distinctively (Baron-Cohen et al., 1987; Grossenbacher, 1997; Steven and Blakemore, 2004; Simner et al., 2006, 2009; Ward et al., 2006; Sinha, 2009; Amin et al., 2011; **Figure 2**).

An analogous ability in the general non-synesthetic population is discussed in philosophical debates such as that of whether there are high-level properties in the content of perception (Siegel, 2006; Bayne, 2009). Those who think that there are such properties in the content of perception hold that high-level properties enter into phenomenal contents. Another debate pertains to *cognitive phenomenology*, according to which phenomenology extends beyond the sensory domain (Pitt, 2004; Bayne and Montague, 2011). This means that phenomenal consciousness cannot be reduced to non-conceptual contents of perceptual states representing sensory properties. Phenomenal experiences also involve various cognitive states including high-level conceptual contents. Apparently, not only non-conceptual attributes determine the character of our phenomenology; conceptual representations also affect our phenomenal experiences. Thus, both perceptual and cognitive states are causally and explanatorily relevant in elucidating how we experience the world because they both exhibit their own phenomenal characters**—***something it is like to be* in such a state for the subject (Strawson, 1994; Chalmers, 1996; Montague and Bayne, 2011; cf. Nagel, 1974), e.g., both seeing an object of deep red color and understanding a complicated mathematical formula are associated with certain phenomenal qualities.

Here we have shown that synesthesia combines a rich diversity of phenomenal contents—a multitude of experiential levels that are primarily driven by: sensory, motor, bodily, emotional or conceptual systems. Consequently, the full combined experience of synesthesia exhibits a holistic epistemic unity—an overall integration of experiences driven by different mental faculties, cognitive and sensory, which are then bound into a coherently unified conscious experience (Mroczko et al., 2009; Mroczko-Wa˛sowicz, 2013). In this way, synesthesia transgresses the boundaries between perception and cognition more than any other mental phenomenon—i.e., between capacities traditionally considered as two independently operating domains (Mroczko-Wa˛sowicz and Werning, 2012).

The cross-domain experiences should not be seen only as an exception, as if it would only be the case for the extraordinary phenomenon of synesthesia. Rather, they should be seen as a general rule applicable also to non-synesthetic practices. If this premise is accepted, synesthesia may be understood as making use of mechanisms similar to those present in ordinary perception and imagery (Ward et al., 2007). The reason why a synesthete takes an extra step in creating a unique synesthetic experience may lie in the nature of the individual semantic network of concepts s/he is deploying. As Jürgens and Nikoli´c (2014) proposed, the semantic structure of each inducing category (e.g., a letter of an alphabet) may be richer for a synesthete by one attribute than for a non-synesthete. Thus, besides the size, shape, or position in the alphabet, or an associated sound, a synesthete would have one more attribute for each grapheme—its color.

In general, we propose that perception and conceptual knowledge are intimately linked and should be investigated together as a unified research problem. By doing so, more progress may be made in understanding human cognition—both synesthetic and non-synesthetic. Thus, lessons learned from synesthesia may be extended into the "ordinary world" of perception and help us develop an integrative approach. This approach would recognize that the boundary between cognition and perception is elusive: perception is more sophisticated than traditionally assumed, rising up to the "high-level". And similarly, "low-level" cognitive operations are fundamentally grounded in our perceptual capabilities (Schyns et al., 1998; Goldstone et al., 2010; Goldstone and Hendrickson, 2010).

Much evidence exists that perception affects cognition such as belief acquisition and formation of concepts (Prinz and Barsalou, 2000; Goldstone et al., 2008; Barsalou, 2009, 2012). This supports the hypothesis of *concept empiricism*, according to which conceptual representations and abstract conceptual knowledge are perceptually based. The underlying operations involve re-activation of perceptual and motor representations (Goldstone and Barsalou, 1998; Barsalou, 1999; Prinz, 2002, 2005). Conceptual contents affect not only the phenomenal character of cognitive states (Montague and Bayne, 2011), but they can equally so inform and influence the phenomenal character of perceptual states; in other words they can cognitively penetrate our perception (Macpherson, 2012). For example, changes in the conceptual content of the inducing stimulus (e.g., the mood or new knowledge about the stimulus) modify perception of the synesthetic concurrent.

This relationship between perception and conception is an integral part of the practopoietic theory and specifically, of the presumed mechanisms of *anapoiesis* responsible for extraction of knowledge from long-term to working memory (Nikoli´c, 2014). In this system, the process of perception and the process of conceptualization of the stimulus rely on the same underlying mechanisms. That is, there is little difference between perceiving a situation or an object (e.g., a chair) and conceptualizing that object, including the most general aspects such as the means of interacting with that object (e.g., knowing what movements are needed in order to sit on a given chair).

Also, our proposal, that semantic mechanisms may be responsible for generating some forms of synesthesia (i.e., an ideasthesia) is compatible with *cognitive penetrability of perception*, a recent interdisciplinary approach based on the presumption that there are various ways in which conscious perception can be modified by cognition–i.e., by thoughts, beliefs, desires, judgments, intentions, moods, emotions, expectations, knowledge, previous experiences and memories (Frith and Dolan, 1997; Bar, 2003; Raftopoulos, 2005, 2009; Vuilleumier and Driver, 2007; Stokes, 2012; Wu, 2013; Vetter and Newen, 2014). In other words, conceptual contents of higher cognitive states not only have causal influence on the contents of perception, but they are explanatorily relevant when trying to account for the processing of perceptual systems. Similarly relevant are the semantic aspects of inducers for explaining the induction of concurrents in synesthesia. A full understanding of how the mind works requires considering the tight relations holding among the cognitive and perceptual domains and their mutual impact.

#### **ACKNOWLEDGMENTS**

The authors would like to thank Uta M. Jürgens for preparing **Figure 2**. This work was supported by Hertie Foundation and Deutsche Forschungsgemeinschaft.

#### **REFERENCES**


neuroanatomical investigations. *Brain Res. Cogn. Brain Res.* 16, 232–237. doi: 10. 1016/s0926-6410(02)00278-1


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 04 December 2013; accepted: 24 June 2014; published online: 19 August 2014*. *Citation: Mroczko-Wa˛sowicz A and Nikoli ´c D (2014) Semantic mechanisms may be responsible for developing synesthesia. Front. Hum. Neurosci. 8:509. doi: 10.3389/fnhum.2014.00509*

*This article was submitted to the journal Frontiers in Human Neuroscience*.

*Copyright © 2014 Mroczko-Wa˛sowicz and Nikoli ´c. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

## Toward a visuospatial developmental account of sequence-space synesthesia

## *Mark C. Price1\* and David G. Pearson2*

*<sup>1</sup> Psychology Faculty, University of Bergen, Bergen, Norway*

*<sup>2</sup> School of Psychology, University of Aberdeen, Aberdeen, UK*

#### *Edited by:*

*Beat Meier, University of Bern, Switzerland*

#### *Reviewed by:*

*Michelle Jarick, MacEwan University, Canada Ilaria Berteletti, Northwestern University, USA*

#### *\*Correspondence:*

*Mark C. Price, Psychology Faculty, University of Bergen, Christiesgate 12, 5015 Bergen, Norway e-mail: mark.price@psysp.uib.no*

Sequence-space synesthetes experience some sequences (e.g., numbers, calendar units) as arranged in *spatial forms*, i.e., spatial patterns in their mind's eye or even outside their body. Various explanations have been offered for this phenomenon. Here we argue that these spatial forms are continuous with varieties of non-synesthetic visuospatial imagery and share their central characteristics. This includes their dynamic and elaborative nature, their involuntary feel, and consistency over time. Drawing from literatures on mental imagery and working memory, we suggest how the initial acquisition and subsequent elaboration of spatial forms could be accounted for in terms of the known developmental trajectory of visuospatial representations. This extends from the formation of image-based representations of verbal material in childhood to the later maturation of dynamic control of imagery. Individual differences in the development of visuospatial style also account for variation in the character of spatial forms, e.g., in terms of distinctions such as visual versus spatial imagery, or ego-centric versus object-based transformations.

**Keywords: synesthesia, sequence-space synesthesia, spatial form, visuospatial, mental imagery, individual difference, development, working memory**

In sequence-space synesthesia, sequences members such as calendar units or numerals are felt to occupy locations within an explicit spatial layout of sequence members – a *spatial form* – that resides in imaginal or extra-bodily space, and ranges from simple 2D to intricate 3D arrangements (Cytowic and Eagleman, 2009; Simner, 2009; Price and Mattingley, 2013; Jonas and Jarick, in press; Price, in press). Typically, the majority of a synesthete's spatial forms originate in childhood. While understanding the development of non-numerical sequence knowledge is in its infancy (Berteletti et al., 2012), even less is known about the developmental origins of sequence-space synesthesia, despite a common incidence [though population estimates vary widely from 2% (Brang et al., 2010) to 20% (Sagiv et al., 2006)]. Some suggest unusual brain processes are involved, perhaps mediated by exaggerated cortical connectivity (e.g., Hubbard et al., 2005, 2011; Tang et al., 2008; Eagleman, 2009). This view sometimes links spatial forms to normal processes of implicit spatial representation of magnitude (Hubbard et al., 2005, 2011; Tang et al., 2008; Simner, 2009). Others suggest spatial forms are continuous with standard varieties of visuospatial mental imagery and best understood as a residue from childhood strategies for encoding abstract verbal sequences (Galton, 1880, 1881; Phillips, 1897; Seron et al., 1992; Sagiv et al., 2006; Price, 2009, in press; Jonas et al., 2011; Price and Mattingley, 2013).

In the latter vein, we offer here a speculative summary of how initial acquisition and subsequent elaboration of spatial forms could be accounted for in terms of the known developmental trajectory of visuospatial representations, drawing from the interconnected literatures on mental imagery and visuospatial working memory.

## **SEQUENCE-SPACE SYNESTHESIA IS A VARIETY OF VISUOSPATIAL IMAGERY**

Sequence-space synesthesia is characterized by internally generated visual and/or spatial sensory experience, and has many experiential similarities with non-synesthetic imagery. We therefore suggest that it is *per definition* a variety of visuospatial imagery, and an example of the considerable individual differences in vividness and prevalence of mental imagery among the general population (McKelvie, 1995; Borst and Kosslyn, 2010). Visuospatial imagery occurs in many contexts, both normal and clinical, but its basic neurocognitive substrates are conventionally taken to be the same across domains. On grounds of explanatory parsimony, we therefore also suggest that standard mechanisms for the production and transformation of conscious visuospatial representations are at very least involved in mediating spatial forms, perhaps even sufficient (Price and Mattingley, 2013; Price, in press). Consistent with this, sequence-space synesthetes show above-average (although not exceptional) non-synesthetic visual imagery at a subjective level (Price, 2009; Rizza and Price, 2012; Meier and Rothen, 2013), and possibly also above-average visuospatial skill at a behavioral level (see Price, in press), including the ability to retain other synesthetes'forms more accurately than nonsynesthetes can (Brang et al., 2010). Furthermore, at least some behavioral correlates of sequence-space synesthesia can be mimicked by use of controlled visuospatial imagery in non-synesthete participants (Price, 2009).

## **EARLY ACQUISITION OF EXPLICIT SPATIAL FORMS**

At a general level the use of spatial representations to help categorize the world is acknowledged as important in children's learning (e.g., Namy et al., 1997; Schwartz and Heiser, 2006)

"fnhum-07-00689" — 2013/10/25 — 14:17 — page 1 — #1

and the ability to construct abstract spatial representations from verbal symbols is a cornerstone of human spatial cognition (Pazzaglia et al., 2012). For example, there is some evidence for an imagery-based "mental time line"in adult participants where temporal events are spatially mapped (Arzy et al., 2009). Heine et al. (2011) have also presented EEG data from elementary school children indicating visual imagery activity during numerical comparisons.

More specifically, initial learning of the sequential ordering of acquired verbal symbols such as numerals or calendar units will benefit from visuospatial representations because, prior to around 7 years, children tend to show little use of their phonological loop to rehearse verbal sequences in working memory (Flavell et al., 1966; Hitch et al., 1988). Whether spatial forms develop may therefore depend on individual differences in the ages at which a given child learns such sequences relative to maturation of their phonological rehearsal. However, even when phonological rehearsal is available, visuospatial representations will benefit learning and long-term retention due to the documented advantages of dual coding (Paivio, 1971; Clark and Paivio, 1991). Apparent randomness and irregularity in the layout of some spatial forms may partly derive from acquisition at these early ages when executive control over image generation is underdeveloped, and when spatial layout is less constrained by social conventions of, for example, left to right writing or circular clockwise time representation.

Persistence of visuospatial representations into later childhood and adulthood will be influenced by continued individual differences in reliance on verbal versus non-verbal strategies. In this respect it is notable that adult sequence-space synesthetes report prevalent general visual imagery, may show above-average visuospatial skill in some domains, and seem to adopt more visuospatial strategies (e.g., during some types of mental arithmetic; Ward et al., 2009). Seron et al. (1992) also reported below-average scores on self-report measures of verbalizing tendency although this was not replicated by Meier and Rothen (2013).

However it is important to acknowledge that spatial forms probably derive from multiple developmental influences. If they resulted solely from a developmental delay in phonological rehearsal, we would expect sequences that are learned earliest in life to be most prone to elicit spatial forms. But this is not the case. Numbers 1–30 are on average acquired by children before the calendar sequence of 12 months, and can be expressed earlier in a linear spatial layout (see Table 1 in Berteletti et al., 2012). On the other hand, number forms appear to be much less common than month forms, occurring largely in people who also have other types of form such as months (Phillips, 1897; Price et al., 2009). Therefore, although a minority of synesthetes may develop all their spatial forms due to problematic rote verbal rehearsal in childhood, additional factors must be at play among the majority of synesthetes. For example, it is possible that that calendar sequences are more difficult to learn than numbers, and so are more sensitive to deficiencies in verbal rehearsal. Alternatively, the normal tendency for linear spatial coding of numbers to precede and later generalize to other sequences such as months (Berteletti et al., 2012) may be disrupted, perhaps again owing to deficient verbal encoding, leading to atypical visualization strategies.

## **FURTHER ELABORATION OF SPATIAL FORMS**

Moving beyond the advantages of visuospatial representations for initial acquisition of sequences, calendar sequences may induce spatial forms more often than numbers because they are more likely to be taught diagrammatically in the first place. Although most synesthetes forget the origins of their forms, some do claim their calendar forms were influenced by exposure to diagrammatic representations from school, TV, calendars, etc. In addition to such environmental influences, continuing developmental incentives for spatial forms may include more complex representational roles such as depicting the cyclical nature of calendar sequences via closed circles.

Once established, spatial forms are reported to increase in complexity from childhood to adulthood (Phillips, 1897; Morton, 1936). This complexity can include creative symbolism such as bends in number lines at decade breaks, distortions of date lines to mark personally significant events, and the use of spatial extent in month or weekday forms to represent the personal importance of some time periods (Price and Mattingley, 2013). By contrast to earlier randomness in forms, controlled and complex imagery of this kind, along with improved processing of spatial relations between objects, will be more dependent on attentional executivebased processes (Pearson et al.,1996; Pearson,2007) which develop markedly from the age of six onward (Gathercole et al., 2004; Chevalier et al., 2013).

As the ability to generate and transform spatial forms matures, they will evolve many characteristics that are continuous with standard controlled visuospatial imagery. These include elaboration of the forms (e.g., from simple spatial trajectories of sequence members into ribbon or tube-like structures), integration of other associated visual imagery into the form, the continued growth of existing forms such as personal time lines, the development of entirely new forms, and the ability to dynamically transform viewpoint such as zooming into the form or navigating within or around the form (Price and Mattingley, 2013; Price, in press). Skill in shifting one's attentional window on the form may be exploited by those synesthetes who claim to use their calendar forms as associative mnemonic systems such as weekly or yearly diaries, or use historical time lines to encode autobiographical and biographical dates (Simner et al., 2009). In a minority of unusual synesthetes who report a great number of forms, visualization strategies appear to extend beyond classic sequences to provide general classification systems or mnemonics for shoes sizes, the Lord's Prayer, etc. Just as self-report measures reveal a wide distribution in the vividness of the mental imagery experienced by non-synesthetes, so the salience and vividness of spatial forms seems to run in graded fashion from the most exotic forms, via vague semi-voluntary forms, to no experience of forms at all.

### **INVOLUNTARY FEELING AND CONSISTENCY ARE NOT UNIQUE TO SYNESTHETIC IMAGERY**

Although the common claim that sequence-space synesthesia is strongly automatic has been challenged (Price and Mattingley, 2013), spatial forms are nevertheless usually reported to have an involuntary feel. Additionally, although spatial forms do appear to evolve over time, their relative stability as long term

"fnhum-07-00689" — 2013/10/25 — 14:17 — page 2 — #2

representations is one of their notable characteristics (Jonas and Jarick, in press). Involuntary feeling and consistency over time have traditionally been taken as central hallmarks of synesthesia. However even these properties of spatial forms are shared by non-synesthetic visuospatial imagery.

In both healthy and clinical populations, imagery experiences commonly result from involuntary as well as deliberate recall processes (Pearson et al., 2013). Regular experience of recurrent involuntary memory is widespread, and occurs for positive and neutral memories as well as negative ones (Berntsen and Rubin, 2008). Pearson et al. (2013) argue that intrusive mental imagery also plays an important role within clinical psychopathology and is associated with many disorders, including post-traumatic stress disorder and social phobia. In addition, according to the Elaborated Intrusion theory of craving (Kavanagh et al., 2005), mental imagery plays a key role during craving for addictive substances and is a hallmark of the phenomenology associated with craving (Andrade et al., 2012). Intrusive imagery may often seem spontaneous, but arises frequently and with involuntary feel merely due to its ease of retrieval when activated by cues via the normal mechanisms of associative retrieval. These cues may be either external (Keane et al., 1991; Carlesimo, 1994) or internal (Pearson, 2012; Pearson et al., 2012; Krans et al., 2013) in nature. Ease of retrieval also accounts for easy voluntary activation of such images. By analogy, spatial forms seem frequently activated in an involuntary manner by relevant cues (e.g., a month name, or thinking generally about the year calendar), as well as easily activated in a more strategic manner (e.g., strategic memorization via associative placement of retrospective or prospective events within a spatial form, illustrated by exceptional date memory among synesthetes with forms for historical time; Simner et al., 2009). Note that Price and Mattingley (2013) suggest voluntary activation of spatial forms plays a critical role in several experimental paradigms which have been claimed previously to demonstrate the automaticity and inflexibility of synesthetes' spatial associations.

The content of recurrent intrusive imagery is usually very consistent in terms of what is depicted in the image (Engelhard et al., 2011; Schulze et al., 2013). Alongside ease of access, consistency is also a characteristic of the often complex visuospatial imagery that may develop as a mnemonic strategy among nonsynesthetes, including professional mnemonists. Importantly, effective mnemonic imagery needs to be consistent and wellestablished in long-term memory in order to provide a stable framework for encoding of to-be-remembered material (e.g., Paivio, 1971; Maguire et al., 2003).

### **INDIVIDUAL DIFFERENCES IN SPATIAL FORMS REFLECT SEPARABLE SUBCOMPONENTS OF VISUOSPATIAL IMAGERY**

In the attempt to establish sequence-space synesthesia as a legitimate and defined topic within cognitive neuroscience, the heterogeneity of spatial forms has tended to be overlooked. Spatial forms vary not merely in their vividness, personal importance, complexity, and involuntariness, but also in their spatial frame of reference, in whether they feel like static depictive visual images or like spatial models, and in the types of spatial transformation that can most naturally be applied to them (Price, in press). Aspects of this rich variation can be naturally accounted for in terms of the separable subcomponents of normal visuospatial imagery, and individual differences in the skills mediated by those subcomponents.

For example, the term "visuospatial imagery" blurs the partial separability, at experiential, behavioral, and neural levels, between "visual" versus "spatial" aspects of visuospatial imagery (Mazard et al., 2004; Kosslyn et al., 2007). While incompletely understood, this distinction refers respectively to a more holistic encoding of the visual or"depictive"appearance of imaged entities, versus more explicit representation of the relative spatial positions of objects or object parts (Hegarty, 2010). It is claimed that individuals tend to be either *object visualizers*, favoring a more visual style of imagery that is associated with vivid high resolution imagery and superior performance on visual memory tasks, or *spatial visualizers*, favoring good spatial analysis and dynamic image transformations such as mental rotation (Kozhevnikov et al., 2005, 2010; Blajenkova et al., 2006). A distinction between object visualizers, spatial visualizers, and verbalizers is also claimed in in the development of cognitive styles among 8- to 17-year olds (Blazhenkova et al., 2011).

There is ongoing debate over whether the experiential character of spatial forms and the behavioral skills of sequence-space synesthetes correspond more closely to the traits of object or spatial visualizers (see Price, in press). For example three studies now show that, for self-report measures, it is in the visual rather than spatial domain that people with "spatial" forms show above-average scores (Price,2009; Rizza and Price,2012; Meier and Rothen, 2013; see also Eagleman, 2009). However our observation of synesthetes' descriptions of their experiences is that spatial forms can reflect characteristics of either trait – i.e., an emphasis on visual detail as seen from one egocentrically defined external viewpoint, or on spatial locations of sequence members in a spatial map that can be navigated as an immersive mental model. One solution to this paradox is that a subset of synesthetes with a spatial visualizer trait have not been detected by studies with small samples or averaged data. Another solution is that visual versus spatial imagery are practically interdependent, even if conceptually separable. For example, a spatial model may be more complex and abstract than a visual image, but visual imagery skill may remain important to instantiate particular views of the spatial model (Pazzaglia et al., 2012). Alternatively, a spatial form could start life as a visual image but develop over time into a spatial model. There is a salient parallel here with literature on human spatial navigation: when people construct spatial representations from verbal input, a hierarchical developmental progression is reported from reliance on landmarks (i.e., an initial focus on specific views), to routes (i.e., a progression through landmarks), to a more flexible survey description (i.e., a spatial model; Nori and Guisberti, 2003). Furthermore, adults seem to show preference for one or other of these levels of spatial representation (Pazzaglia et al., 2000) which, like the overlapping distinction between object/spatial-visualizers, may map onto typologies of spatial form.

For those spatialforms which reach the developmental sophistication of a spatial model, their dynamic transformation is another area where individual differences can be informed by fractionation

"fnhum-07-00689" — 2013/10/25 — 14:17 — page 3 — #3

of visuospatial imagery skills. Some synesthetes report objectcentered transformation – e.g., as the months go by, their spatial model of the calendar months rotates around or in front of them. Others report ego-centric transformations – e.g., they feel they navigate through, or around, their spatial model. Price (in press)suggests that synesthetes' preferred mode of transformation may be related to known individual differences in object- versus ego-centric transformation skill which rely on distinct neural processing resources and are partly independent at a behavioral level, as revealed by visuospatial psychometric tests (Zacks and Michelon, 2005). Interestingly, a classic distinction in spatial metaphor for time among the general non-synesthete population is between ego-moving metaphors (observer feels they move along a time line toward the future) and time-moving metaphors (time is felt to move like a river past the observer; Clark, 1973; Lakoff and Johnson, 1980; Gentner et al., 2002). This raises the possibility that spatial forms and non-synesthetes' spatial metaphors both reflect similar individual differences in aptitude for one or other mode of visuospatial image transformation.

A practical implication of these kinds of individual differences is that visuospatial skills will differ among synesthetes. Unless synesthetes' individual profiles are carefully aligned, group comparisons of their behavioral skills versus control samples stand in danger of confusion and inconsistency.

## **SPATIAL FORMS AND IMPLICIT SPATIAL REPRESENTATION OF MAGNITUDE**

Our approach differs from suggestions (Hubbard et al., 2005, 2011; Tang et al., 2008; Simner, 2009) that spatial forms are an abnormally explicit expression of the analog spatial representations which represent magnitude, not just of numbers but across many continuous perceptual dimensions (A Theory of Magnitude, ATOM, Walsh, 2003). As argued by de Hevia et al. (2006), this use of space as a general analog continuous representation for magnitude, exemplified in the concept of the implicit Mental Number Line (MNL), can be contrasted with the ordering of discrete items of information (e.g., symbolic labels) in representational space. Indeed, neuropsychological data show representation of magnitude and ordinality to be dissociated, even though they interact

#### **REFERENCES**


numerical and non-numerical order in children. *Cognition* 124, 304–313. doi: 10.1016/j.cognition.2012.05. 015


(Berteletti et al., 2012). We suggest that a fundamental problem in attempting to align spatial forms with systems for magnitude representation is that most spatial forms express non-magnitude sequences, such as calendar units or the alphabet, unlike numbers which possess cardinality as well as ordinality. Moreover, in spatial forms, the use of spatial extension to represent personal importance rather than temporal length of certain months illustrates that magnitude representation is not their main function. Rather, they are easily activated, long-term, visuospatial representations of ordinality which aid some individuals to acquire, retrieve, and mentally navigate within abstract sequences, which can be elaborated to symbolize further information, and which may sometimes provide a facilitative template for further associative memories.

## **CONCLUSION**

Central hallmarks of sequence-space synesthesia, including involuntary feel, consistency over time, dynamic nature, and elaborative imagery, are not unique to this variety of imagery, but shared by everyday visuospatial imagery, intrusive imagery, and mnemonic imagery. Furthermore, a visuospatial developmental perspective illuminates why and when sequence-space synesthesia originates in some people, and why its character varies between synesthetes. We therefore suggest strong continuity between synesthetic spatial forms and normal processes of visuospatial imagery.

Interest in the relationship between mental imagery and synesthesia is by no means confined to sequence-space synesthesia, and there is growing evidence that other varieties of synesthesia are also associated with above-average visual imagery (Meier and Rothen, 2013; Price, in press). However evidence also suggests that predisposition to experience this variety of synesthesia is separable from other types of synesthesia (Novich et al., 2011), and that it is associated with a different grouping of cognitive styles compared, for example, with grapheme-color synesthesia (Meier and Rothen, 2013). The extent to which our current perspective may extend to other varieties of synesthesia therefore remains an open question.

#### **ACKNOWLEDGMENT**

This work was supported by a Norwegian research council (NFR) Grant No. 710028 to Mark Price.

*Psychol. (Hove)* 63, 2031–2050. doi: 10.1080/17470211003802459


"fnhum-07-00689" — 2013/10/25 — 14:17 — page 4 — #4

T. Vecchi (Amsterdam: John Benjamins), 156–171.


"fnhum-07-00689" — 2013/10/25 — 14:17 — page 5 — #5


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 31 July 2013; accepted: 30 September 2013; published online: 25 October 2013.*

*Citation: Price MC and Pearson DG (2013) Toward a visuospatial developmental account of sequence-space synesthesia. Front. Hum. Neurosci. 7:689. doi: 10.3389/fnhum.2013.00689*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2013 Price and Pearson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, providedthe original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

"fnhum-07-00689" — 2013/10/25 — 14:17 — page 6 — #6

## An extended case study on the phenomenology of sequence-space synesthesia

## *Cassandra Gould1,2,3\*, Tom Froese1,2,4,5, Adam B. Barrett 1,2, Jamie Ward1,6 and Anil K. Seth1,2*

*<sup>1</sup> Sackler Centre for Consciousness Science, University of Sussex, Brighton, UK*


*<sup>5</sup> Centro de Ciencias de la Complejidad, Universidad Nacional Autónoma de México, Mexico, Mexico*

*<sup>6</sup> School of Psychology, University of Sussex, Brighton, UK*

#### *Edited by:*

*Beat Meier, University of Bern, Switzerland*

#### *Reviewed by:*

*Claire Petitmengin, Institut Mines-Télécom, France Mark Price, University of Bergen, Norway*

#### *\*Correspondence:*

*Cassandra Gould, Sackler Centre for Consciousness Science, University of Sussex, Falmer, Chichester 1, 164 Brighton, East Sussex BN1 9RH, UK e-mail: c.d.gould@sussex.ac.uk*

Investigation of synesthesia phenomenology in adults is needed to constrain accounts of developmental trajectories of this trait. We report an extended phenomenological investigation of sequence-space synesthesia in a single case (AB). We used the Elicitation Interview (EI) method to facilitate repeated exploration of AB's synesthetic experience. During an EI the subject's attention is selectively guided by the interviewer in order to reveal precise details about the experience. Detailed analysis of the resulting 9 h of interview transcripts provided a comprehensive description of AB's synesthetic experience, including several novel observations. For example, we describe a specific spatial reference frame (a "mental room") in which AB's concurrents occur, and which overlays his perception of the real world (the "physical room"). AB is able to switch his attention voluntarily between this mental room and the physical room. Exemplifying the EI method, some of our observations were previously unknown even to AB. For example, AB initially reported to experience concurrents following visual presentation, yet we determined that in the majority of cases the concurrent followed an internal verbalization of the inducer, indicating an auditory component to sequence-space synesthesia. This finding is congruent with typical rehearsal of inducer sequences during development, implicating cross-modal interactions between auditory and visual systems in the genesis of this synesthetic form. To our knowledge, this paper describes the first application of an EI to synesthesia, and the first systematic longitudinal investigation of the first-person experience of synesthesia since the re-emergence of interest in this topic in the 1980's. These descriptions move beyond rudimentary graphical or spatial representations of the synesthetic spatial form, thereby providing new targets for neurobehavioral analysis.

**Keywords: sequence-space synesthesia, phenomenology, elicitation interview, conscious experience, case study**

## **INTRODUCTION**

In sequence-space synesthesia (SSS) (also known as spatial-form synesthesia), ordinal sequences such as numbers, letters and calendar months are experienced as occupying precise locations in extended areas of space (Price, 2009; Ward, 2013). Despite affecting up to 20% of the population (Sagiv et al., 2006; Mann et al., 2009), our understanding of the visual phenomenology of this experience is largely limited to schematic spatial representations of concurrents spatial-forms (see for example **Figure 1**).

Some of the most complete introspective reports of SSS were collected from 26 SS synesthetes, via a questionnaire with both open-ended and "yes/no/maybe" questions (Seron et al., 1992). This investigation of Seron et al. uncovered a number of features afforded little attention in subsequent research. For example, one subject indicated that certain sections of their number form had a black background and white digits, whilst other sections had a white background and black digits (see **Figure 2**). Nineteen subjects reported that once "activated," a concurrent could not be moved (Seron et al., 1992, p. 174). Subjects also reported being better able to "make use" of their representation where they had a free "visual field," e.g., when their eyes were closed, and that the "vividness" of the number concurrent increases when the subjects concentrate on it (Seron et al., 1992). One subject reported automatically experiencing the concurrents immediately surrounding the induced item, although these adjacent numbers were "a bit less salient" (Seron et al., 1992, p. 180). Seron et al. re-ignited interest in the exploration of SSS, but concluded that the "genuineness of the number representations" remains the primary open question (Seron et al., 1992, p. 185). Twenty years later, the majority of published research on SSS continues to be concerned with establishing the reliability and behavioral impact of SSS. For example, SSS has been shown to reduce reaction times to items presented in positions congruent to a concurrent spatial location (e.g., Jarick et al., 2009a). However, there has been little regard for

*<sup>2</sup> Department of Informatics, University of Sussex, Brighton, UK*

the phenomenal aspects which may shed light on other processes known to be modulated by SSS, e.g., visuospatial working memory, visuospatial imagery, sequence representation and memory for personal or historical dates (see Price and Mattingley, 2013 for a review).

The Elicitation Interview (EI, also known as the Explicitation Interview) technique is derived from the phenomenological tradition of Husserl and the study of phenomenological content of conscious experience (Vermersch, 1994). The EI was formulated specifically for the purposes of obtaining descriptions of prereflective experience (that is, aspects of our experience which we are not aware of, prior to active reflection upon them), whereas a semi-structured interview is more generally applied to gather information on the reflective experience of the subject. In performing an EI, the trained interviewer guides the introspection of the subject by directing their attention to precise aspects of their phenomenological experience (Vermersch, 1994). This model of pre-reflective access is based on the premise that guided introspection facilitates the transformation of implicit to explicit. Commentary has been provided elsewhere on whether EI itself changes the experience, rather than enabling increased reporting access to the experience (see Froese et al., 2011a). This debate is primarily a philosophical issue of deep versus shallow concepts of lived experience and is outside the scope of this paper.

EI has previously been applied to increase awareness of epileptic prodrome features (Petitmengin et al., 2007), to understand the adaptation strategies of elderly drivers (Cahour et al., 2010), to provide introspective access to unconscious decision making processes (Petitmengin et al., 2013), and to develop and identify distinct "attentional states" before the onset of a cognitive task (Lutz et al., 2002) [see also Maurel (2009) for other applications and Froese et al. (2011a) for a review of recent research using second-person methods]. A full and detailed method for the conduct of an EI has been presented elsewhere (e.g., Petitmengin,

2006). Below we provide a brief overview of the assumptions and priorities in EI data collection, to highlight the unique properties of this method and how it may be applied beneficially in the investigation of synesthesia phenomenology.

During the EI, we aim for the subject to reach and maintain an "evocation state" through interviewer-led direction of attention. Evocation is a deep introspective state in which the subject is in "direct contact" with their experience, and re-living it as they speak. A positive evocation state is suggested by slow and considered speech, avoidance of generalizations, a distant or unfocused gaze and often the use of language in the present tense (the subject may say, "I can see *...*" rather than "I remember seeing *...*"). There is some similarity between the induction of the evocation state and that of a hypnotic state. The interviewer may use "Ericksonian" language to refocus the subject and gain access to pre-reflective experience, for example when the participant responds, "I don't know *...*," the interviewer may ask, "How do you know that you don't know?" (Petitmengin, 2006). Although the original event may only have lasted a second, through EI we are able to prolong and extend the experience of that event during the evocation state, through subtle use of language and cues. A potential risk in the collection of such first-person data of this kind is the unintentional retrospective transformation of events, as demonstrated by experimental false memory induction (e.g., Loftus and Pickrell, 1995) and false recollection (Greenberg, 2004). Petitmengin et al. (2013) have recently demonstrated that the process of becoming aware of previously pre-reflective aspects of experience does not necessarily lead to confabulation. The specific language used by participants in reporting false memories will, however, allow a skilled interviewer to recognize infidelity in the report and exclude the relevant section from analysis (Froese et al., 2011a). Other bias effects are reduced in the normal practice of EI by monitoring and maintaining the evocation state of the subject, to ensure they are reliably verbalizing the experience and not their beliefs about the experience. Active and open questioning, with rephrasing of the subjects own statements, serves to draw attention to specific aspects which may generally be overlooked in normal discourse. Examples of good practice in bracketing to reduce bias effects and maintaining evocation are provided in the Materials and Methods section.

We present here an extended phenomenological description of SSS in a single case. We apply the EI technique, and demonstrate that qualitative second-person techniques can be used to gather first-person information about a perceptual experience of SSS, which may be used to guide future qualitative and quantitative investigations. The assumptions of EI make it particularly suited to dissecting processes such as SSS, which appear to occur automatically or with little conscious effort, but which also retain a cognitive "anchor" or event upon which to ground the interview. Steady-state experiences such as synesthesia may also engender high fidelity reports, as the event can be reported immediately as it unfolds, rather than retrospectively. The results of the EI were analyzed following Petitmengin (2006), involving successive iterations of aggregation and abstraction of themes, schemas or structures of the experience. This process is similar to the more widely practiced Grounded Theory analysis of qualitative data (Glaser and Strauss, 1967). The themes uncovered via this method have been collated and organized into a detailed and expandable html mind map, which we have made available to readers via the Supplementary Materials. This paper, supported by the Supplementary Materials, presents converging evidence on the reproducibility of the Seron et al. phenomenological investigations. Importantly, we extend the level of phenomenological detail acquired and describe novel aspects of the SSS experience. We demonstrate that a more complete understanding of SSS phenomenology may shed light on features of the experience such as the spatial reference frame in which the concurrents exist, along with issues relating to the apparent automaticity of concurrents and vividness of spatial-forms, which in turn may speak to the recent reframing of SSS as relating to enhanced visual imagery (Price and Mattingley, 2013). We also demonstrate that greater detail in our understanding of the phenomenological experience of SSS can reveal subtle idiosyncratic aspects of the SSS experience, which may contribute to our understanding of the development of this trait.

## **MATERIALS AND METHODS**

The subject (AB) is a 30 year old, right handed male with no medical history of psychological or neurological trauma. AB reported to have experienced SSS for numbers, days of the week and months, for as long as he could remember. Prior to interview, no facet of AB's SSS was identified as particularly remarkable or out of the ordinary. To test the consistency of his spatial-forms, AB was asked to reproduce his number forms at the start of data collection and again 22 months post interview. These two schematics are sufficiently similar to confirm the genuineness of AB's synesthesia (see **Figure 1**).

## **CONDUCT OF THE INTERVIEWS**

Interviews were conducted by a trained and experienced EI practitioner. Nine hours of interview were conducted over nine non-consecutive sessions spanning 2 months. The interviews were conducted in a private room and recorded following the informed consent of the subject. The interviews were exploratory and involved a fluid examination of topics of interest as they arose. Inducers were presented audibly, for example the interviewer would ask "If I say, five *...* What do you see?" (20100215\_79- 81)1. A visually presented inducer was used on three occasions (20100408\_3, 20100408\_65, 20100415\_19), with the inducer drawn on a page of the interviewer's notebook. The nature and purpose of the EI method was defined for the participant at the beginning of the first interview session. The interviewer highlighted the requirement for the subject to describe the experience in detail, and reminded the participant of the interviewer's role in guiding and maintaining the focus of attention, suggesting that these particular skills may develop as the series of interviews progress (20100215\_2). EI is more commonly applied in the exploration of past events, either naturally occurring or triggered for the purposes of the interview. In the present investigation, experiences were triggered by the interviewer, during the interview itself, and then immediately described and re-evoked. This deviation from the standard EI practice enabled the free

<sup>1</sup>Throughout the Materials and Methods and Results sections, we have made use of direct quotations from interview transcripts, in order to convey the essence of the interview conduct and mode of questioning. Quotations are referenced with the interview date in the format YYYYMMDD and the paragraph number in the transcript.

exploration of the SSS experience, rather than limiting the interview content to a single event. Each interview session began by revisiting the features of interest in the previous interview, or addressing any topics which had been raised informally between interviews.

#### *Maintaining the subject's focus*

The interviewer used active questioning and reformulation of the subject's own words in order to maintain attention on a single instant or aspect of the experience, as exemplified in the questioning style demonstrated below.

AB: *Ahm ... I'm looking at it ... at the moment I'm projecting it onto the screen in front of me and just ... It's all on that screen.* INTERVIEWER: *So if you look at that screen ...* (20100215\_ 45-46).

Here the subject refers to concepts of "projecting" and "screen," both of which have not been explored at this stage. In order to gain a description of what the subject means by the use of these words, the interviewer simply asks AB to look at the screen and describe what he sees.

AB: *So, so you can sort of see a floor underneath the five, and then there's kind of a wall ... ahm* (20100222\_20, 20100222\_22). INTERVIEWER: *And if you focus on the wall and the floor, and try to keep your focus there, how does the periphery look?* (20100222\_32).

Here the interviewer uses the terms "wall" and "floor" as the subject did. This reuse of the subjects own words serves to maintain and focus his attention on these specific features.

#### *Bracketing for bias reduction*

"Bracketing" interpretations of events is conducted during the interview and during analysis, to minimize the interpretive bias. Data were removed from analysis if the interviewer's questioning was deemed to be leading. Examples of bracketed or excluded data are given below for clarity (emphasis added).


In (1) the interviewer identified a "belief" or theory that the subject held about his experience. The subject says, "I think I'm *...*," demonstrating that he is trying to infer or interpret what he is doing, rather than simply describing the experience. He interprets this action as "projecting." "Projecting" is a term often used in the context of synesthesia to refer to the relative location of concurrents, in external or peripersonal space, and as such may be used by the participant as an analogy or "short-cut" to describe a projector-like experience. The interviewer chose to immediately divert from this description and return to the topic at a later time2 . In (2) the interviewer allowed their own beliefs about the experience to enter the discourse. The term "latching" is not used by the subject himself at any point in the interview. The line of questioning suggests that the interviewer formed a belief regarding attachment between the concurrent and another object in earlier interview sessions, however, this is the only time in which the belief is imposed upon the subject. This section of interview and all related discourse were removed from analysis.

## **ANALYSIS**

Interview data were transcribed and then processed through successive rounds of coding, aggregation and abstraction, as illustrated in **Figure 3**. Data analysis was conducted blind (that is, not by the interviewer), by a researcher trained in the EI method with no prior theoretical knowledge of SSS, as suggested in Froese et al. (2011b). Blind analysis enabled the independent observation of any interviewer bias and ensured that the themes devised were are only those which are present in the data.

Interview audio files were transcribed verbatim by contracted professionals, including stammers and interjections. Transcribers noted changes in pace, intonation or volume of speech, where specific emphasis is placed on words, and significant or lengthy pauses, to assist in the monitoring of evocation state. Transcribers were blind to the nature and purpose of the interviews. Fidelity of transcription was checked against a random sample of audio files. Transcribed data were assessed for validity according to the principles of EI, by confirming that the report related to the experience as it happened and did not contain comments or thoughts about the experience on the part of the participant or interviewer. Reports were also examined for consistency over the interview sessions. Any reports found to be incompatible with the EI requirements or inconsistent across sessions were excluded from analysis.

The complete set of interview transcripts were read through by the analyst, to identify initial themes or topics. Themes were then summarized into short statements and a code devised to represent the theme. The code was subsequently attached to relevant sections of the transcript using QSR NVivo 9 software (QSR International, 2010). Once all data relevant to a particular theme had been coded, the theme summary was revised and new subtheme codes were devised. A second round of coding was then conducted using sub-themes.

The coded transcript was exported with paragraph references into Freemind mind mapping software (FreeMind Project, 2010), clustered by themes and sub-themes. Individual excerpts were systematically arranged into appropriate headings (with further subheadings generated where necessary) to ensure all references to a topic were collated. The tree-like representation of the mind-mapping software enabled a systematic analysis of the temporal development of events through vertical arrangement of branches (diachronic structure), and aggregation of simultaneously occurring aspects of the experience (synchronic

<sup>2</sup>Note that term "projecting" was also bracketed in the earlier example (1. Maintaining the subject's focus) as the interviewer pursued a line of questioning relating to the known object of the "screen."

structures) under a single heading. Ordered clusters were summarized into short sections of prose describing the content and context of the cluster. Abstractions were made for every level of the map from the lowest to the highest level of the theme (for example, the description of the concurrent contexts includes 4 levels of abstraction, over 77 separate sub-headings—see Supplementary Material).

#### **RESULTS**

The EI sessions provide comprehensive descriptions of the firstperson experience of SSS for subject AB, including aspects of which AB was previously unaware. Analysis of EI data identified several themes of potential importance to SSS research. Full detail of each of these themes are provided in an html expandable mind map via the Supplementary Material, with the bottom-most level of each branch containing all transcript references which were collated in the generation of the theme. The themes and descriptions are not exhaustive of the SSS phenomenology for this participant, rather they reflect the developing discourse of the interview sessions.

#### **DISCOURSE CONTENT**

**Table 1** outlines the coverage of individual codes in the transcripts. The majority of the discussion was related to AB's number form (55.8% of all discourse) as opposed to his calendar or alphabet forms (15.5% and 10.3%, respectively). Some of the most informative phenomena were only covered briefly during interview (for example only 1.4% of the transcripts are related to interference from the physical room, see section Mental room), however, when these brief statements are collated and examined in terms of their interactions in the data as a whole, we find them to be relevant to much of the synesthetic experience.

## **BASIC FORM DESCRIPTIONS**

The forms examined include a spatial number form, mathematical concepts such as pi, an alphabet form, a form for age, a calendar form for years, a form for the months of year and multiple forms for time. These data provide a comprehensive description of the visual synesthetic experience and on the whole are consistent with descriptions obtained by less rigorous methods. This continuity demonstrates that in terms of basic phenomenology, our participant's experience may be considered "normal" for SSS. Brief descriptions of the spatial forms are provided below, with further detail available in the Supplementary Materials.

#### *Number form and mathematical concepts*

Number concurrents generally appear as black Arabic symbols in an Arial-like type face. Certain numbers (notably five and fifty six) are described as having "metallic" (20100222\_247) or "velvety" textures (20100222\_321) or surface patterns such as "cross hatching" (20100408\_67) or pockmarks, "as if the ink has come away in lots of places" (20100320a\_246). The texture or patterning may move around and change although the number itself remains static.

Both fractions and decimals are viewed in the correct position relative to their surrounding numbers. Different notations

#### **Table 1 | Coding summary report.**


*Major themes are given in bold font, with sub-themes in italics. Coverage represents the percentage of all interview data which were coded to a particular theme or sub-theme. A reference may be coded to two or more items when a particular section of discourse is relevant to a number of different themes, thus the total coded coverage does not sum to 100%. Number of occurrences gives the number of separate references to a theme or sub-theme within the interview data, with each instance being variable in length from a single line to an extended discussion. As the interviews were exploratory, the coverage does not necessarily reflect the content of general SSS experience, rather it relates to the content of the interview.*

of fractions can be generated intentionally (e.g., 1.5 may be represented as 3/2 or 6/4)and appear as though superimposed on top of each other. Large numbers (in the thousands) appear with the comma separator.

When discussing mathematical concepts, multiple defining equations and graphs are involuntarily experienced following a related inducer. For example, when discussing Planck's constant the subject reports, "I got e equals H bar times F *...*. And just underneath I've got H bar times a Greek nu as well [*...*] I've got H bar equals H over 2 pi." (20100320a\_219-224). These concurrents include visual experiences of letters and numbers to multiple decimal places, and associated graph forms. For example, INTERVIEWER: "What about e? AB: So I've got a flash of the exponential curve and then I saw the letter e, it took a while for the equal sign and numbers to appear but then they are there." (20100320a\_71-72). "INTERVIEWER: If you go back to e, do you see the two graphs that you mentioned? AB: Yeah, kind of one over here and one over there." (20100320a\_137-138).

Numbers such as pi are experienced in their correct position in relation to the integer on the number line. A black area or "black hole" (20100229\_200) is experienced for numbers with infinite decimal places, with the decimal numbers trailing toward the black hole, "sucking up all those infinite digits *...* just sort of get them out of the way" (20100229\_200). The black hole is flat and AB can change his perspective to view it from behind (20100229\_122).

#### *Alphabet form*

AB's synesthetic form for the alphabet appears as capital letters (20100329\_215), in an Arial-like type face (20100329\_337- 339) with a "ghost-like" translucency (20100408\_313) and a reproducible spatial relationship between alphabet items. Specific patterns or textures are generally not reported with the letter form, however, following inspection of the letter B it is described as though "it's made of ribbons attached to each other" (20100408\_329).

#### *Ages*

AB experiences concurrents similar to his number form when exploring concepts of age. AB reports, however, that the experiences of age and number concurrents are differentiated because numbers (such as 28) are more often used in the context of age rather than mathematics (20100320b\_90).

The concurrents elicited by inducer concepts of prenatal ages (conception to birth) are accompanied by static visual scenes of embryonic development to a hospital scene with a woman in labor representing birth (20100320b\_ 103-138). The description of the scene for birth (age zero) contains doctors and nurses in a hospital room, medical implements on a tray, storage drawers and a work surface, a woman lying on a bed, a window which looks onto the hospital corridor and electrical equipment over the bed (20100320b\_153). AB reports that he can hear the sound of a baby crying from somewhere outside the scene but no other sounds from the hospital room itself (20100320b\_139-149). When AB is asked if there is anything "before" that scene (equivalent to negative numbers) he reports a "vague sense of something in the womb" (20100320b\_110) but with an unknown or indistinct location. When the interviewer induces "6 months pregnancy" AB is able to identify the location of this age and reports "a sort of fairly black womb with a sort of like fetus in it" (20100320b\_112). AB also describes other scenes in separate spatial locations, including conception "at the very top" as a "magnified picture of an egg with loads of sperm" (20100320b\_127), then a fertilized egg and a "clump of cells" (20100320b\_114) followed by further cell divisions in the embryonic development. Zero and one years of age are separated in space to accommodate "all those first months in the first year of life" (20100320b\_182). When directed to the zero time point, AB reports images of a new-born baby "wrapped up in white blankets", and at 6 months old AB reports hearing a baby cry (20100320b\_192).

The interviewer proposes that the first year might contain images of a baby "crawling around on the floor, or something like that" (20100320b\_195). At this stage AB remarks that he is beginning to experience "a few more general images" (20100320b\_196), including the memory of an occasion where he saw a friends child at one year old. AB spontaneously offers that these images are quite distinct from those described in relation to prenatal age, as he is positioned within the scene and it is surrounding him (20100320b\_196), where as in the pictorial representations of age AB was not himself a feature of the scene.

## *Calendars*

AB reports a spatial form for calendar aspects such as years, months and time. His year form is similar to his number form, but there is no comma separator. The inducer "one thousand, nine hundred and ninety five" leads to the experience of a number concurrent, whereas the inducer "nineteen ninety five" leads to a year concurrent (20100320a\_237). AB is able to describe and draw his spatial form for the more abstract year inducer of 4000 BC, with an inflection point between BC and AD.

AB's spatial form for months includes the experience of digits and a line arranged circularly, with August at the top and subsequent months arranged clockwise, with February at the bottom. Within the months themselves each 10 day segment follows a specific direction on the line such that AB can identify a date not only by the position of that month in relation to the others but also by the direction of the numbers on the line surrounding the date.

AB reports a clock concurrent induced by the time of day. His synesthetic clock takes multiple forms: (1) an analog clock face with hour and minute hands, minute markers and larger hour markers; (2) an orange digital clock with a flashing semicolon for seconds; (3) a separate digital clock with a different type-face. In directing his attention to aspects of the analog clock, AB is able to induce the experience of digits on the clock face where there were none before. AB reports the apparent motion of a second hand as it flashes "in and out of existence" (20100320b\_16-18) each time his attention returns to it.

#### **THEMES WITHIN THE SSS EXPERIENCE**

The themes identified in the EI go beyond basic form descriptions of as they address precise aspects of the experience and how it unfolds. Six main themes were identified from the interviews, as illustrated in **Figure 4**.

#### *Mental room*

AB describes his synesthetic percepts as existing in a "mental room" (20100320a\_108) as opposed to the physical room, which refers to the room in which the interview was conducted and which supports the normal (non-synesthetic) visual experiences (20100408\_193). The percepts in the mental room are "superimposed" on the physical room (20100408\_137); they have the same physical location as items in the mental room, but the mental and physical exist as if in two "different dimensions" (20100408\_163, 20100408\_165, 20100408\_189).

*I'm sort of seeing the [physical] room behind [the concurrent]. But then I sort of feel the [concurrent] letters are in a different kind of space as well* (20100329\_235).

*It's almost like you have this whole room and then there's like a copy of this whole room (20100408\_191). And then the pen stays in this [physical] room. But all the [synesthetic] numbers and things are in the copy of this room [the mental room]* (20100408\_193).

AB reports that he can switch his attention between the physical and mental rooms by forced effort (20100320a\_108) or defocusing his eyes and "looking into space" (20100408\_33). AB retains an awareness of the physical room whilst looking at the synesthetic percepts (20100320a\_112) although the physical room appears "out of focus" (20100320a\_114), similar to a normal experience of an item in peripheral vision (20100320a\_114).

*Erm, well I can just alternate my attention between the physical room and the mental room I've got* (20100320a\_108).

*I guess all of the physical room is like erm, like what's in my periphery normally. So the focus of my eyes is somewhere about here, because there's nothing in the physical room here. The whole physical room is out of focus* (20100320a\_114).

*INTERVIEWER: So if you focus your attention on the [mental room] ... your visual awareness of the [physical] room fades out ...? AB: Yeah. It never leaves me completely, the physical room* (20100320a\_111-112).

Concurrents tend to be located in areas of space where there is little interference from physical objects, such as the top of an empty table (20100329\_89) or in an area of clear space (20100329\_107, 20100415\_238).

*If my eyes are kind of focused on some point in space in which there is no physical object, then it's easier [for the concurrent to appear]* (20100408\_31).

*If there's an object in this [physical] room, in the same place as one of the numbers are in the other [mental] room, then I see the object rather than the number* (20100408 195).

*The floor in this [physical] room is obscuring the floor in that space [the mental room]. I can't really see down* (20100415\_278).

The interference of visual percepts in the physical room upon synesthetic percepts in the mental is also reported in the visual presentation of an inducer, where AB reports having to look away from the "distracting piece of paper" (20100408\_13) in order to experience the synesthetic form.

*INTERVIEWER: What do you see? [Shows AB a letter H drawn on a notepad] AB: I see the letter H on the page. INTERVIEWER: So what do you need to do in order to see the alphabet line? AB: Erm, have to look away from the image. I can see them here* (201004 08\_3-5).

*When I was focused on the paper there was nothing else except what was physically there (20100408\_15). It's almost like the actual physical H can't be part of the mental alphabet* (20100408\_23).

*So I see the 4 [drawn in Interviewer's notebook] and then there's nothing really stable coming into view. It's like the number line doesn't know which direction to go in. It's like there are various things down on the table and something over here ... Nothing very precise came in. Just very different from when you are telling me to think of the number 4* (20100408\_65-67).

#### *Concurrent contexts*

AB's concurrents appear in a range of "contexts" which describe the perceptual synesthetic experience around the concurrent in the mental room.

*White page.* The "white page" context is described as an A4 piece of paper, presented vertically in front of AB, forward and below his eye line (see **Figure 5**). The area around the white page has "a purple layer here, a green layer here and some black." (20100329\_395). Concurrents appear centered on the page in a font size which would fill a "good proportion of the paper *...* as if there's only room for that one digit [on the page] with a good margin around the edge" (20100215\_146-148). The white page provides contrast for the concurrents to be viewed against the background of the physical room, as AB reports that he has difficulty identifying the concurrents when they haven't got the white page behind them:"It's almost better to have the white page there, because it helps me to see the A better" (20100329\_399, see also 20100415\_43).

*Corridor.* When viewing a section of the number line (as opposed to individual concurrents), concurrents appear in a "corridor" (see **Figure 6**). The walls of the corridor are "quite opaque (20100329\_42) although "not like a hard surface like the table, if you put anything behind [the wall of the corridor] it would go into the fog and you wouldn't be able to see it" (20100408\_349). These walls are described relatively consistently as a "orangey, reddy, greeney color" (20100222\_148) elsewhere, "orangey, browney, greeney color" (20100222\_103) and "greenish, reddish, orangey" (20100229\_108). The floor of the corridor has similar opaque characteristic as the walls (20100222\_31, 20100222\_186). The corridor has a ceiling (e.g., 20100222\_117) and on two occasions the ceiling is referred to with a source of light, with "little circles of light going down the corridor" (20100222\_159) and opening up into a "Sun roof"

(20100222\_326). The corridor itself has a "kind of a yellowish light kind of permeating it" (20100222\_146).

The induced concurrent is reported to "float" in the middle of the corridor (20100320a\_47) whilst the concurrents immediately surrounding it are against the wall or just outside the corridor.

Concurrents with multiple decimal places are experienced with the decimals trailing down the length of the corridor. The size of the corridor appears to vary. On one occasion the corridor is reported to be not much bigger than AB's head (20100222\_126), on another it is as wide as AB is tall (20100229\_182). The corridor appears to expand width ways to encompass more numbers in the form: "It's kind of like the corridor wants to enlarge and have the 7 over here" (20100329\_107).

*Free space.* The free space context is identified by reference to a specific coloring of the background and a lack of white page or corridor features. AB describes the free space background as "unstable," "sometimes it's purplish and sometimes it's greenish." (20100329\_13). The background "doesn't make the [physical] room colored. It's just like another world [mental room] that I can focus my attention on, or I can bring my attention back to the [physical] room" (20100415\_55).

*Line.* AB reports a "line" feature in his number and calendar forms. The line is not present in the alphabet form as, "[the alphabet] has no continuous values" (20100329\_306). The line is seen both in the free space context, and in the corridor of the number form. The line is black and has a 3D shape like a "very long rectangle" (20100222\_58). The line is below the white page in the vertical plane such that AB has to look down from the white page to see the line (20100215\_153). AB suggests that he can see the page and the line at the same time, although his language use is slightly ambiguous and may suggest that in switching his attention between the line or page contexts, his perception of the other is diminished:

*I can definitely see it being kind of located below where the page was* (20100215\_155) [emphasis added].

INTERVIEWER: *Can you see the number line at the same time [as the white page]? AB: Yeah, when I see the number line it's going like this and below* (20100229\_49-51) [emphasis added].

In the corridor context, the base of the line is "resting on the floor [of the corridor]" (20100222\_58), bisecting the corridors width, "going into the wall of the corridor" (20100229\_165-166), and follows the gradient of the number form. In the free space context, the number concurrents "hover" just above the line (20100215\_172) or the line appears with a gap into which the concurrent would fit if viewed in the corridor (20100222\_95), with the gap acting as a "marker for where the [number] should go." (20100222\_109). There is a suggestion that the line functions as a reference for directing AB's gaze or attention to other concurrents, as AB follows it's gradient toward where he "knows" another concurrent to be located (e.g., 20100320a\_298).

*Cave.* The "cave" is the usual context of AB's alphabet form. The cave is described with a "dampness" (20100329\_441) or "muddy" atmosphere (20100408\_315), a dark background (20100329\_426) with depth, like a "a dark hole" (20100329\_365). Letter percepts within the cave are described as having "translucent" (20100408\_315) or "ghost-like" qualities (20100408\_315) rather than a solid form. AB describes his head as being in one wall of the cave, allowing him to look into the depth or toward the entrance. When the entrance of the cave is explored, AB describes it as positioned in the side of a mountain (20100329\_459) with a naturalistic scene outside and a "sense of an expansive landscape" (20100329\_457), "sky and clouds" (20100329\_461).

*Felt presence.* AB occasionally reports that he can sense the presence of a concurrent but not see it directly. These percepts are often described as "vague," for example, "these are bit more vague these ones. They are just kind of more like the feeling or the presence of the three there" (20100415\_27), "I've kind of got like the sense of the six and the seven sort of somewhere around here. Not really quite sure where they are" (20100329\_67).

*Context transitions.* There are dynamic transitions between the contexts. These transitions generally occur automatically and in a predictable manner, although AB is able to trigger them of his own volition to a limited degree. We present the following brief diachronic description of context transitions for a number concurrent (for a more detailed description of each stage in the transition, see the Supplementary Material):

Initially a new number concurrent is presented on the white page. Surrounding the white page is a color which is similar to the free space context. Directed by a change in AB's focus of attention to the area below the page, the white page "melts" from the bottom to the top. This melting is described as, "[if] the page was made out of ice and there was fire underneath it, and it's kind of melting gradually and kind of in a fairly homogeneous way" (20100329\_375). The shift in focus of attention from the page to the line causes a new concurrent to appear above the line. Once the new concurrent is stable, the line and corridor come into view.

#### *Internal and external dialog*

Auditory presentation appears to be the most effective induction method for AB's synesthetic percepts, as it is not associated with direct physical interference of a visually presented item. The audibly induced item is often accompanied by those items surrounding it in sequence (e.g., the induced experience of a two will be accompanied by a one and three in the correct positions). "Thinking about" an inducer will also lead to the experience of a concurrent (20100305\_142), with the thought process often involving inner speech (20100408\_48-51). On other occasions AB reports to visualize a concurrent to induce a synesthetic experience (20100222\_261-263).

*You say 15, I'm actually seeing more digits now. I sort of see 14 and 16 and 17* (20100215\_61).

*So I'm saying M over and over again in my head. But I've got a glimpse of an L at one point. And I've got a glimpse of N O P. It's hard to just look at the M* (20100329\_249).

*INTERVIEWER: So if you look at the M, how many letters are you aware of at the same time? AB: Just depends where I focus my attention. If I focus just on the M it's quite hard. It's almost like I have to utter the names of the other letters in my head in order to see them. But as soon as I think, you know, looking at the M and then thinking about what else I can see, need to start uttering M or L and then they come into being* (20100329\_244-245).

*INTERVIEWER: So what if you pick a random one, just choose it yourself, how would you go to it? AB: Yes, then I would have to say it in my head. So I think Q was kind of here. ... I kind of just thought of the letter Q by saying it in my head, then I kind of looked for it* (20100408\_48-51).

#### *Directional attention between concurrents*

If AB allows his eyes to move freely along the number or alphabet form, he can see as many or as few concurrents as he chooses (20100329\_251). Directing attention may also involve inner speech, for example AB reports in one instance that during this act of "looking for a concurrent" he says to himself: "right I'm going to look for the 4 now!" (2010040 8\_138).

*INTERVIEWER: if you let your gaze move a little bit more freely, how easy is it for the other letters to appear? AB: It's pretty easy for H I J K L M N O P Q to appear. R S T. All I have to do is move my attention along the thing. I can kind of see the whole alphabet if I want to* (20100329\_250-251).

*INTERVIEWER: What about ... the other end of the alphabet, Z? AB: Just have to re-look where it is. INTERVIEWER: It's just a movement of the eyes? AB: Yeah. INTERVIEWER: You don't have to say anything. AB: No, I think you saying it is enough* (20100408\_ 42-47).

AB also has a strong prior knowledge of where each concurrent should be located in relation to another in the same form. Redirection of attention based on prior knowledge also involves inner speech.

*Erm, normally it's like I said to myself, "OK, it's a three so down to the left of three there should be a one and two." And then, well there they are* (20100408\_93).

### *Single inducer leads to multiple concurrents*

A single inducer may lead to the experience of multiple representations of the same concurrent, particularly if AB is directed to perform some act upon that concurrent. For example, when AB is asked to view a four from a different angle, he reports to experience two fours, one in the normal, face-on view and one back-to-front (20100408\_125-129). New or alternate representations of the same concurrent occupy different locations in the mental room.

*There's a four and a six kind of here. ... INTERVIEWER: If you moved your head from side to side like this, how does your perception of the number change? AB: Erm, sometimes it tilts a little bit so it's kind of bi-stable. ... It's like there's two here. So there's one where the four is just like that, and then is another one where it's kind of [back to front]. INTERVIEWER: I've noticed that at the moment you are actually standing still. Can you see both of the different ones even though you are not moving? AB: Yeah, there's kind of one like this and there's one kind of like that. INTERVIEWER: So you have two different forms there? AB: Yeah* (20100408\_125-129).

#### *Associations lead to involuntary inductions*

Concurrents may be experienced involuntarily if they are related to the induced concurrent through normal associative mechanisms, for example when "a physical constant [*...*] like the speed of light" (20100320a\_195-196) is presented as an inducer, AB reported the involuntary experience of five concurrents associated with other physical constants (20100320a\_195-224). Similarly, when AB is asked to transform a letter into an italic font, a bold font (an associated type-face) is perceived (20100329\_340-345).

*INTERVIEWER: What about a physical constant ... Like the speed of light (20100320a\_195-196). AB: Yeah, what else came into view was Planck's constant, like the H bar. I just saw like an H bar and then over here there's an equals and a C squared came up (20100320a\_202). INTERVIEWER: So how come the Plank's constant appeared? ... Did you think of the Plank's constant as well? AB: Yeah, I guess because it's another fundamental constant of the universe. It's like C [the speed of light]* (20100320a\_211-214).

*INTERVIEWER: Do you think you could get them in italics? AB: Erm, saw the C though going to italics very briefly, the capital C. ... The lower case C is kind of being mixed up. It's kind of in bold font. <laughter>*(20100329\_340-345).

#### *Concurrent experiences persist*

Once induced, the experience of the concurrent was found to persist without the need for it to be directly referred to (20100229\_329, 20100408\_40-41). Similarly AB reports that a percept may remain stable even if it is taken out of view or attention is directed elsewhere; once attention is returned, the concurrent is described as if it existed independently of AB's attention (20100305\_131-132, 2010022 9\_388).

*60 has rooted now because we've been talking about it for quite a bit* (20100229\_329).

*INTERVIEWER: When you look at the ... alphabet [form], are you aware of having to say H in order to keep it in focus? AB: No, I think once it's there I don't have to say H* (20100408\_40-41).

*INTERVIEWER: What if you ... take them out of view completely so if you look behind you so immediately you don't see either of them and then you turn around again. Do they remain exactly where they were before? AB: Yeah still five, six [56], 57* (20100305\_131-132).

After several concurrents have been induced through the interview process, AB's experience contains multiple independent concurrent items, for example, INTERVIEWER: "Do you have any number lines ready that we could do that with? AB: Erm, I've got all sorts of number lines" (20100408\_386-387). This persistence in the visual representation and spatial location of a concurrent caused experimental difficulties where AB found there were too may concurrents present to concentrate on a single item, causing him to feel "bogged up with loads of numbers" (20100229\_384). This experience of multiple concurrents was not explored further.

#### *Perspective on percept*

Concurrents which are directly induced are generally presented in front of AB in the correct perspective, i.e., "facing," although there is one report of a concurrent being turned away slightly (20100415\_185-187).

*I'm kind of seeing the five right in front of me* (20100215\_170).

*So I've sort of got the number five to the right of my focus now, and kind of in front of my focus* (20100222\_33).

New concurrents may also appear at some distance particularly if they are a new representation of the original concurrent.

*INTERVIEWER: So the zoomed out version of the alphabet disappeared when you switched your attention to the A? AB: ... it's kind of in the distance. It's probably further away from me than I can see* (20100329\_280-281).

AB is able to view concurrents from different perspectives, e.g., if they are placed behind him or if he physically moves round or above them.

*So this one is directly facing me and that one is kind of facing that way. So I'm kind of slightly sideways on. ... if you were in the chair one to your right, it would face you* (20100415\_185-187).

*INTERVIEWER: Can you [physically move round to look at the percept from different angles]? AB: I can see the back of the four. Yeah I can kind of see the four backwards. Sometimes it sort of spins around* (20100408\_104-105).

#### *Stability of percept*

AB often reports "instability" in his percepts, as a flashing or brief appearance of the concurrent. This instability is reported when AB attempts to modify a feature of the concurrent from its preferred or "default" form. For example, attempting to change the type-face (20100329\_340-345) or perspective of the alphabet form (20100415\_97-113) results in an experience of a percept flickering between the default and requested view/feature.

*[From an upper case A] I can get the lower case A to flash [into an upper case A], but then [it] doesn't stay there* (20100329\_323).

*[AB is asked to see what happens if he looks at a concurrent whilst he changes his viewing angle by standing up] So it's kind of hard to stand up slowly enough ... INTERVIEWER: ... you're trying to stand up slowly in order to see what? AB: ... where some kind of change happens. ... it seems to flicker (20100415\_97-101). It just came into view when I sort of reached there at a certain angle.*

*INTERVIEWER: Is it not on the table right now? AB: It's sort of flickering* (20100415\_107-109).

Where AB's knowledge of a number sequence is limited (e.g., in the decimal places of pi), he only experiences stable concurrents for those numbers of which he is certain (20100229\_132).

*So now I've got 3.14159 dot dot dot eight, six, five, six, and then a whole pool, a whole load of numbers just kind of like jumbled up, like kind of just there, not really knowing where to go* (20100229\_147).

*So it's unstable after 3.141527986. Seems pretty unstable after the third or fourth decimal point. ... it's not really holding still it's kind of changing because of other numbers trying to get in from above. ... So it's 3.14 and one and a five, and that's where I start getting difficulties ... there's a nine above the two. INTERVIEWER: How is it above the two? AB: Just directly above on the page. INTERVIEWER: Is it stable up there? AB: I just see a jumble of numbers* (20100229\_71-83).

*INTERVIEWER: What do you see after the five? AB: It's kind of dot dot dot, and then there's kind of some rather blurry numbers to the right in a crowd* (20100320a\_14-15).

Selectively directing AB's attention to specific features of a concurrent appears to improve the stability of the percepts and AB's ability to describe it, for example he often uses phrases such as, "if I think a little bit more then I see *...*" (20100222\_5) or "if I really try and focus *...*" (20100229\_398) before going on to describe a feature which was previously "unstable."

*[AB was asked to inspect the texture of the 5 concurrent in detail] Yeah. Erm but nothing that stays really too stable. If I really try and focus I don't know, I see a little white so it's kind of mainly black top bit of it. Erm, then I am seeing a little white circles ... [there is a] white circle missing from top corner as though someone has punched a hole out of it from the top corner* (20100229\_398) [emphasis added].

#### *Voluntarily controllable acts*

AB is able to perform a number of acts to modify his synesthetic experience. These include:

1. Changing perspective on a concurrent, either by physically moving around the room or adopting a different mental perceptual position such that he can view the concurrent from a different angle. *INTERVIEWER: ... So how does it change when you stand up for example? AB: ... looks like I am kind of above it, kind of like this. It's kind of got some depth to it this time* (20100415\_68-69).


Although AB's reports would suggest that he can in some cases modify a concurrent itself, attempts to do so lead to the generation of a new concurrent with the desired characteristic, whilst the original remains unchanged. AB's ability to exert some level of control over his synesthetic experience demonstrate the dynamic and elaborative nature of the visual experience.

## **DISCUSSION**

In this paper we used the EI technique to provide a detailed phenomenological analysis of SSS in a single case. The conclusions drawn are appropriate to our participant and further work will be required to test their applicability to a wider population. Some external validation of the themes derived can none the less be achieved, by reference to the existing (limited) descriptions of phenomenology. Where there is continuity between the reports of our participant and those provided elsewhere, we may assume that these particular themes are generalizable to a wider population. As our participant reports common phenomenological details in the basic form descriptions, it is likely that his experience in general is also typical of a wider SSS population. Therefore, where other more novel aspects of phenomenology are derived for this participant, it is possible that these too may be generalizable.

Our results show that AB's synesthetic concurrents exist in what he terms a "mental room" and that he is able to selectively shift his attention to either the mental room of the concurrents or the physical room in which the interviews were conducted. Although the mental and physical rooms co-exist, percepts in the physical room appear to interfere with synesthetic concurrents in the mental room. This interference effect was also found to be a feature of visually presented inducers such that AB had to divert his gaze from the physical inducer in order to perceive the concurrent. The generation of a concurrent frequently followed auditory induction, for example AB uses internal verbilisation of the inducer name in order to perceive the concurrent, even when the inducer was presented visually. We also found that concurrents are automatically experienced by association to the inducer, for example the discussion of the physical constant for the speed of light causes the automatic perception of related physical constants. AB's concurrents are not limited to simple letters/numbers, but also include equations and graph notations. The induced item is often perceived with adjacent items on the number line, for example the induction of a two may lead to the perception of a one and a three. The stability of synesthetic percepts is found to increase with focused attention on the concurrent percept. AB is also able to voluntarily modulate the features of a concurrent, although they appear to have a default type-face and/or perspective.

As this is a case study, it remains to be seen to what extent the precise nature and content of all themes of AB's SSS experience generalize to a wider population. For example, the experience of a "black hole" representative of infinite decimal places may be highly idiosyncratic and unique to AB in his conceptualization of infinity. Other idiosyncratic features may include AB's ability to move around the spatial form whilst it remains static, and the precise nature and content of the concurrent contexts of the white page, corridor and cave. The unique and artificial experience of the EI may have drawn out details of AB's experience which are not a common feature of the every-day SSS experience. For example, on many occasions AB only perceives the attended concurrent, and a few surrounding items. This may be due to the high level of focus placed on individual concurrents for the purposes of guided exploration. Further investigation may determine whether the process of EI can reveal similar themes for other participants. Three themes in particular may be relevant in developing our understanding of SSS, namely, (1) the description of the physical and mental room; (2) the effect of internal and external dialog; (3) the apparent persistence of concurrents after induction.

#### **PHYSICAL-MENTAL ROOM**

AB's description of the mental-room can be considered in light of the projector and associator distinctions in grapheme-color synesthesia (Dixon et al., 2004). AB reported in the first interview that he is "projecting a mental image" when he perceives a synesthetic concurrent (20100215\_22). AB's description of the mental room as "another dimension" (20100408\_185) is interesting, but of itself says little about the experience. More telling is the apparent relationships between the physical and mental rooms such that the physical can obscure or distract from the mental and make it difficult to visualize or fully induce a concurrent. This effect may be informative in providing a basis for the exploration of first-person experience of projector-associator effects, however, the projector-associator distinction is an assumption of sequencecolor synesthesia, and not SSS; although SSS experiences have been described as occupying "distinct loci in imaginal, peripersonal or extrapersonal space" (Price and Pearson, 2013, p. 1166), projector-like and associator-like traits have not, to the best of our knowledge, been formalized in SSS. Any direct comparison between the experience of our subject and the current understanding of the projector-associator experience may therefore be premature. The observation that AB can readily distinguish between the physical and mental rooms (even when they interfere) underlines that his synesthetic experiences, while at times perceptually real, are not subjectively veridical—that is, they are not experienced as being part of the external, real world (Seth, 2014).

#### **INTERNAL AND EXTERNAL DIALOG**

AB internally verbalized the name of the inducer each time he selfinduced a concurrent (i.e., when the inducer was not presented by the interviewer). This included occasions when he redirected his attention from one concurrent to another, or when he freely chose a concurrent to explore. When he was visually presented with an inducer, he needed to say the name of that inducer internally before the concurrent was experienced, and repeated the name of the inducer to stabilize his visual experience and maintain his focus of attention. This use of inner speech may not be surprising, given the inter-relation of orthographic and phonemic representations in neural processing, however, it was uniquely identifiable by the process of EI and has not been previously demonstrated in SSS; other methods may have accepted AB's initial descriptions (e.g., of the redirection of spatial attention) without probing further into the acts involved in the generation of a concurrent. This is an example of EI enabling reflective access to an aspect of subjective experience which was previously pre-reflective. This also exemplifies the in-depth exploration of diachronic structures of experience promoted by EI.

External auditory instruction was the most direct and effective route to induction of a concurrent for AB. The primacy of auditory induction may account for AB's primary use of inner speech during all other inductions. Auditory-visual synesthesia is commonly considered to include only more overt forms such as the experience of colors with music (e.g., Goller et al., 2009). As such, SSS and other forms (e.g., grapheme-color synesthesia) have typically been assumed to result from visual-visual interactions. There are, however, investigations of visual-visual types which have used auditory presentation of inducers and found results comparable to visually presented inducers (Paulesu et al., 1995; Nunn et al., 2002; Steven et al., 2006). In SSS, a case has also been described in which auditory and visual presentation of an inducer lead to different perspectives of the form being perceived (Jarick et al., 2009b). These findings suggest that auditory induction is sufficient to cause the synesthetic experience, and the evidence here from EI suggests that it is in most cases necessary in the case of AB, even when items are presented visually.

Assuming there is some overlap between the neural mechanisms involved in processing internal versus external speech, the processes of concurrent induction via inner speech may be similar to the processes involved in concurrent induction via external auditory presentation. As all methods of induction engaged inner speech for this subject, the experience of SSS might be re-framed as primarily auditory-visual process for AB and potentially in other SS synesthetes. Compatible with this hypothesis, children in most English-speaking countries learn the ordinal sequence of the alphabet phonemically through song before they are able to associate the visual grapheme and letter name (Ehri, 2009). Seron et al. found that the SS synesthetes they investigated were less efficient in verbal strategies according to Paivio's degree of imagery questionnaire (Paivio and Harshman, 1983) and suggest that the visual spatial form may have developed as a compensatory strategy to reinforce the ordinality of audibly presented items (Seron et al., 1992). This suggestion may have particular implications for the development of SSS, particularly if it is framed as an auditory-visual association, rather than a visual-visual or visual-spatial.

This formalism of SSS as a trait relevant to the development of visuospatial representations has been re-introduced by Price and Pearson (2013), who argued that the apparent automaticity of spatial forms may in fact represent deliberate recall of the specific visuospatial representation. Indeed, Price and Pearson (2013) suggest that the development of the spatial form may be closely related to the development of the phonological loop during rehearsal, where the dual encoding of visuospatial and auditory representations may be beneficial to learning. Our data support the central intuitions of Price and Pearson (2013) and Price and Mattingley (2013), in the suggested involvement of the phonological loop in concurrent generation and by demonstrating that the apparent automaticity of our participant's experience is graded, or only has the appearance of an involuntary feel.

#### **VISUOSPATIAL MEMORY AND PERSISTENCE OF CONCURRENTS**

Once induced, AB will continue to experience a concurrent even when direct attention is drawn away from it. Similarly, Jarick reported that her subject (L) experienced a persistence of induced concurrents until they were "not needed any more" for the purposes of mental viewpoint navigation (Jarick, 2010, p. 88). Although this finding of persistence has also been identified via more commonplace first-person methods (e.g., semi-structured interviews), the level of detail made available by EI is unparalleled thus far. Specifically, whilst Jarick refers to persistence until concurrents "are not needed" (Jarick, 2010), our data suggest that the development of this persistence is based on sustained exposure and focus on the concurrent, which may have implications for the development or encoding of spatial forms in terms of visuospatial imagery and memory. We also find that AB experiences a sense of "presence" when he cannot directly see a concurrent but knows where it should be in relation to others. These two features suggest involvement of visuo-spatial memory processes in SSS, where the memory of a concurrent position is sufficient to hold the synesthetic experience in active working memory, either as a visual experience or as non-spatial visual content. This feature may be relevant to the development of SSS in individuals, as it may enable the maintenance of a visual representation when ordinal sequences are being taught. Brang et al. (2010) and Simner et al. (2009) have reported superior visuo-spatial memory abilities in SS synesthetes, as demonstrated in their ability to learn new spatial forms (Brang et al., 2010) and perform above average in assessments of visuo-spatial recall such as the Visual Patterns Test (Della Sala, 1997; Simner et al., 2009).

#### **BLIND ANALYSIS EXTENDS PREVIOUS INTROSPECTIVE REPORTS**

Our results extend standard introspective reports collected elsewhere. For example, 19 of the Seron et al. subjects report that, once activated, a concurrent could not be moved, supporting our findings on the persistence of concurrents and AB's necessity to generate a new representation of the concurrent rather than modifying an original. Subjects also reported being better able to "make use" of their representation where they had a free "visual field," e.g., when their eyes were closed. This statement is compatible with our findings of physical interference, where AB's concurrents were placed in areas of an uninterrupted visual scene in the mental room, so as to avoid conflict with items in the physical room which hold more salience than the mental room percepts. As with our subject, Seron et al. also report that for his subjects the "vividness" of the number concurrent increases when the subjects concentrate on it (Seron et al., 1992, p. 174). Through the application and analysis of the EI data collected here we have been able to identify that this concentration is experienced by the subject in many cases as internal recitation of the inducer name, highlighting the importance of inner speech in the induction process. One of the Seron et al. subjects also reports to automatically experience the concurrents immediately surrounding the induced item, as with our subject.

#### **NEW INSIGHTS MAY BE USED TO DIRECT FUTURE QUALITATIVE AND QUANTITATIVE INVESTIGATIONS**

The present findings suggest new opportunities for future investigations of SSS and associated synesthesia phenomenology. For example, quantitative investigations should consider the interference of physical objects in the perception of synesthetic concurrents and associated limitations of visually presented inducers.

Early self-reports from SS synesthetes described the experience as occurring involuntarily and automatically (Cytowic, 1989). However, our subject reports that he is not continually "swamped" with concurrents on a daily basis, despite continually reading and seeing numbers and letters in his external environment. Indeed, a pilot investigation with our participant, using the Descriptive Experience Sampling method (Hurlburt, 2009; Hurlburt et al., 2009), captured a moment when AB was actively engaged with letters and numbers on a computer screen, but he reported no awareness of any synesthetic experience. As such, the perception of a concurrent is not entirely automatic for AB, in line with evidence that automaticity of concurrents can vary in grapheme-color synesthesia (Rothen et al., 2013). We have also noted specific instances of where AB does retain voluntary control over his synesthetic experience. These examples support the challenge to the assumption of automaticity in SSS, as suggested by Price and Pearson (2013), and the issue of automaticity in SSS may be a topic usefully explored with EI across multiple subjects. For our subject, the concurrent was generated through internal verbalization; it remains to be seen, with the further application of EI or other procedures, how much detail can be obtained regarding the apparent automaticity of concurrent perception. The role of visual imagery in SSS could also be investigated further (Price and Pearson, 2013). Specifically, the experience of "unstable" concurrents could be investigated in synesthetic participants, and compared qualitatively with descriptions of "unstable" visual imagery in non-synesthetic participants3.

The phenomenology presented here is by no means exhaustive. A number of unresolved issues are immediately identifiable, such as the role of inner speech and focused attention in stabilizing a concurrent. We may ask how inner speech impacts mental imagery, and whether non-synesthetes employ similar methods of inner speech as observed here. We also demonstrate that the apparent automaticity of SSS can be investigated with this method in terms of phenomenological experience. Price and Mattingley (2013) have recently argued that SSS does not meet the necessary behavioral criteria to be termed truly automatic. Indeed, we have determined here that although synesthetic concurrents were initially described to arise automatically, in the majority of cases, our subject went through a process of internal verbalization of the inducer name before the concurrent was perceived. This may relate to the task-specific strategies suggested by Price and Mattingley (2013) to mediate the cueing effects of SSS. Further exploration of the diachronic and synchronic structure of the spatial cueing effect may help determine the causal relationship between the appearance of the concurrent and the shift in spatial attention.

## **CONCLUSIONS**

Phenomenological descriptions of SSS have so far been largely confined to descriptions of spatial locations pertaining to specific inducers (e.g., numbers, letters, calendar date). Here, employing a second-person method designed to increase the reporting access to pre-reflective aspects of experience, we have provided a comprehensive description of our subject's spatial form for a range of ordinal sequences including number, calendars, alphabets and ages, and extend the depth and breadth of relevant phenomenological information which has been acquired for SSS, or indeed any other form of visual synesthesia. We also introduced new associated synesthetic experiences such as concurrents for graphs, equations and mathematical concepts. Using the formalism of Grounded Theory analysis, we identified seven main themes in AB's SSS experience, as below.


Some of these themes have been described elsewhere (e.g., Seron et al., 1992; Jarick, 2010), however, these sources provide only limited information the first-person experience of the subject. As such, the present results provide a unique level of detail which may be beneficial in devising future investigations.

<sup>3</sup>EI could also be usefully applied in elaborating the phenomenological experience of "vividness" of visual imagery, which is central in ratings of visual imagery, e.g., VVIQ (Marks, 1973).

We report the first application of the EI method to SSS, and indeed to synesthesia more widely. We demonstrate that it is possible to gain informative second-person descriptions of synesthetic phenomenology from a naïve subject, through rigorous data collection and analysis, and demonstrate the value of the EI method in enabling access to subtle aspects of the experience. We gained unprecedented detail regarding the precise synesthetic phenomenology of this subject, which may be generalizable to a wider population. We also provide access to interview data as raw transcripts, alongside the structured and analyzed form in the Supplementary Material. This is a rich data source and we invite the interested reader to explore avenues for further connections between the themes uncovered here and other aspects of SSS, or synesthesia more generally. Appreciation for the precise phenomenological experience in synesthesia should focus future research endeavors, as we seek to understand the extent and limitations of the experience from the perspective of the participant. Such work may transform consciousness science by bringing objectivity to experiential phenomenology.

## **AUTHOR CONTRIBUTIONS**

Cassandra Gould, Tom Froese, and Anil K. Seth designed the research. Tom Froese performed interviews with participant Adam B. Barrett and Cassandra Gould analyzed data and wrote the paper. Anil K. Seth and Jamie Ward supervised the project. All authors contributed to the final version of the paper.

## **ACKNOWLEDGMENTS**

The authors wish to thank Claire Petitmengin and Béatrice Cahour (Telecom ParisTech, France) for their guidance in EI training and analysis. Financial support from EPSRC Leadership Fellowship EP/G007543/1 awarded to Anil K. Seth and The Dr. Mortimer and Dame Theresa Sackler Foundation.

#### **SUPPLEMENTARY MATERIAL: HTML EXPANDABLE MIND MAP**

Online supplementary material contains the full set of raw interview transcripts and descriptions of all themes uncovered during EI with this subject. Each theme can be explored by clicking the expand (+) button, revealing a summary description and further sub-themes. The bottom-most level contains verbatim excerpts from the in EI sessions, along with a interview date and paragraph reference.

The Supplementary Material for this article can be found online at: http://www.frontiersin.org/journal/10.3389/fnhum. 2014.00433/abstract

#### **REFERENCES**


Vermersch, P. (1994). *L'entretien d'explicitation*. Paris: ESF.

Ward, J. (2013). Synesthesia. *Annu. Rev. Psychol.* 64, 49–75. doi: 10.1146/annurevpsych-113011-143840

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 01 November 2013; accepted: 28 May 2014; published online: 03 July 2014. Citation: Gould C, Froese T, Barrett AB, Ward J and Seth AK (2014) An extended case study on the phenomenology of sequence-space synesthesia. Front. Hum. Neurosci. 8:433. doi: 10.3389/fnhum.2014.00433*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2014 Gould, Froese, Barrett, Ward and Seth. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Insights from introspection: a commentary on Gould et al. (2014), "An extended case study on the phenomenology of spatial form synaesthesia"

## *Mark C. Price\**

*Psychology Faculty, University of Bergen, Bergen, Norway \*Correspondence: mark.price@psysp.uib.no*

#### *Edited and reviewed by:*

*Beat Meier, University of Bern, Switzerland*

**Keywords: synesthesia, sequence-space synesthesia, spatial form, mental imagery, visual imagery, introspection, automaticity**

#### **A commentary on**

**An extended case study on the phenomenology of spatial form synaesthesia**

*by Gould, C., Froese, T., Barrett, A. B., Ward, J., and Seth, A. K. (2014). Front. Hum. Neurosci. 8:433. doi: 10.3389/ fnhum.2014.00433*

Gould et al. (2014) refreshingly devote an entire paper to investigating the detailed *experience* of a synaesthete (AB), using a rigorous qualitative method. Their paper addresses sequence-space synaesthesia, in which people associate members of sequences such as months or numbers with specific spatial locations that together create an overall spatial pattern or "form" for each sequence. I have previously argued that sequence-space synaesthesia shows strong continuity with normal voluntary visuospatial imagery (Price, 2009, 2013), and may be an extension of normal childhood development of visuospatial skills (Price and Pearson, 2013). Additionally I have suggested that behavioral evidence for the automaticity of this synaesthesia is less robust than often claimed (Price and Mattingley, 2013). Gould et al.'s data provide striking support for these claims.

Take continuity with normal visuospatial imagery:

(1) *Some spatial forms seem indistinguishable from normal imagery:* e.g., AB's multiple and dynamic "spatial forms" for clock-time constitute a vivid visual imagery of clocks that would unlikely be considered synaesthetic if reported by a non-synaesthete.


Next consider how voluntary attention mediates AB's experiences:

(1) Attentional movement is required to activate parts of AB's number line that were not previously in focus.


Gould et al.'s data go beyond merely exemplifying previous claims. AB's distinction between physical space and the "mental room" in which his synaesthesia is experienced is perhaps not so novel as suggested by the authors, because it is pre-empted by earlier self-report surveys (Phillips, 1897; Sagiv et al., 2006) and autobiographical accounts (Tyler, 2005, pp. 34–35), but descriptions of complex interaction between AB's physical and mental spaces seem more detailed than previous accounts. They also challenge simplistic dichotomy between projector and associator synaesthesia. Moreover, the reliance of AB's synaesthesia on internally verbalizing the name of the inducer does seem a novel contribution to our knowledge (and is consistent with suggestions that spatial forms arise when visuospatial codes are adopted to help retention of sequences whose individual members are primarily learned via auditory exposure). Gould et al. stress that aspects of these experiences were previously unknown even to the synaesthete, and that the formal qualitative method of the Elicitation Interview (EI), adopted in their study, can uncover such previously hidden phenomenology. It should nevertheless be noted that more informal introspection can also reveal novel aspects of synaesthetic experience. For example, Tyler (2005, p. 36) describes how he first thought his synaesthetic color associations were limited to numbers, "but then I started running through the letters of the alphabet and realized that, indeed, they did evoke particular colors when I paid attention to this modality." Whether the EI method is necessary for some kinds of discovery, or merely corroborates what can be revealed by more informal introspection, is therefore open to question.

In any case, it follows that introspective studies can transform synaesthetes' experiences rather than merely articulating them in neutral fashion. This violates the definitional synaesthetic criterion of consistency over time, convergent with other recent claims that synaesthetic concurrents can change over shorter and longer time scales (Simner, 2012; Meier et al., 2014). So how do the contents of synaesthetic experience change during detailed introspective exploration? Are the new experiences *de novo* creations, or shifts from preconscious to conscious representations. Alternatively, some synaesthetic experience may initially color what James (1890; Mangan, 1993) referred to as the unattended "fringe" of sensory awareness, but be brought into focal awareness via introspective dialog. Future work needs to empirically distinguish these possibilities.

An obvious question with single-case studies is the extent to which the participant is representative. Gould et al. consider AB a "normal" sequence-space synaesthete as his experiences are consistent with previous reports in the literature (although they also acknowledge that AB's introspective dialog may have modified his experiences beyond what is typical). However, normality is a questionable concept here because the character of this variety of synaesthesia seems to vary among individuals along many dimensions. These include automaticity, complexity, visual vs. spatial quality, projector-associator gradient, spatial reference frame and types of spatial transformation that can be applied (Price, 2009, 2013). For example, AB reports his spatial forms are flexibly placed into an environmental reference frame. By contrast, other synaesthetes report spatial forms that are more egocentrically defined. A thorough taxonomy for spatial forms is still missing from the field, but the type of research provided by Gould et al. helps us map the territory that needs to be included in such a taxonomy.

## **REFERENCES**


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 30 May 2014; accepted: 31 May 2014; published online: 02 July 2014.*

*Citation: Price MC (2014) Insights from introspection: a commentary on Gould et al. (2014), "An extended case study on the phenomenology of spatial form synaesthesia." Front. Hum. Neurosci. 8:439. doi: 10.3389/fnhum. 2014.00439*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2014 Price. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

**REVIEW ARTICLE** published: 28 February 2014 doi: 10.3389/fnhum.2014.00098

## *Marcus R.Watson1, Kathleen A. Akins <sup>2</sup> \*, Chris Spiker <sup>2</sup> , Lyle Crawford <sup>2</sup> and James T. Enns <sup>3</sup>*

*<sup>1</sup> Department of Ophthalmology and Visual Sciences, Child and Family Research Institute, University of British Columbia, Vancouver, BC, Canada*

*<sup>2</sup> Department of Philosophy, Simon Fraser University, Burnaby, BC, Canada*

*<sup>3</sup> Department of Psychology, University of British Columbia, Vancouver, BC, Canada*

#### *Edited by:*

*Beat Meier, University of Bern, Switzerland*

#### *Reviewed by:*

*Jamie Ward, University of Sussex, UK Jonathan Winawer, New York University, USA*

#### *\*Correspondence:*

*Kathleen A. Akins, Department of Philosophy, Simon Fraser University, 4604 Diamond Building, Burnaby, BC V5A 1S6, Canada e-mail: kathleea@sfu.ca*

Learning and synesthesia are profoundly interconnected. On the one hand, the development of synesthesia is clearly influenced by learning. Synesthetic inducers – the stimuli that evoke these unusual experiences – often involve the perception of complex properties learned in early childhood, e.g., letters, musical notes, numbers, months of the year, and even swimming strokes. Further, recent research has shown that the associations individual synesthetes make with these learned inducers are not arbitrary, but are strongly influenced by the structure of the learned domain. For instance, the synesthetic colors of letters are partially determined by letter frequency and the relative positions of letters in the alphabet. On the other hand, there is also a small, but growing, body of literature which shows that synesthesia can influence or be helpful in learning. For instance, synesthetes appear to be able to use their unusual experiences as mnemonic devices and can even exploit them while learning novel abstract categories. Here we review these two directions of influence and argue that they are interconnected. We propose that synesthesia arises, at least in part, because of the cognitive demands of learning in childhood, and that it is used to aid perception and understanding of a variety of learned categories. Our thesis is that the structural similarities between synesthetic triggering stimuli and synesthetic experiences are the remnants, the fossilized traces, of past learning challenges for which synsethesia was helpful.

**Keywords: synesthesia, learning and memory, plasticity and learning, perceptual development, cognitive development, multisensory processing**

#### **INTRODUCTION**

When synesthesia first came under scientific scrutiny in the 1800s, a fairly common view was that synesthetic experiences were learned, and moreover served to help synesthetes with tasks such as mathematics, remembering sequences, and many other learning challenges (e.g., Galton, 1881; Calkins, 1893; Jewanski et al., 2011). After being largely ignored for much of the twentieth century, synesthesia returned to the scientific mainstream about 25 years ago but learning was largely rejected as either a cause or function of synesthesia. For example, Cytowic (1989), who virtually single-handedly re-introduced synesthesia to the modern research community, originally argued that "synesthesia is not learned" and many other researchers were similarly at pains to distinguish learned or remembered associations from "real" synesthesia (cf. Harrison and Baron-Cohen, 1996; Elias et al., 2003; Ramachandran and Hubbard, 2003).

In recent years, however, it has become clear that the earlier viewpoint needs to be taken seriously. There has been a flurry of research on the profound impact of learning on synesthesia, and several prominent researchers now argue that learning and conceptual factors are critical components of synesthetic development (cf. Simner et al., 2009a; Jürgens and Nikolic, 2012; Deroy and Spence, 2013a; Witthoft and Winawer, 2013). And some of those who formerly denied the role of learning appear to have reversed their stance (cf., Cytowic and Eagleman, 2009). There has also been a smaller body of literature dealing with influences

in the other direction: the utility of synesthesia for learning new material.

These two streams of research are the focus of the first two sections of this review. In the final section, we present a theory of synesthetic development that is motivated by our interpretation and integration of the literature in both streams. Here we argue that synesthesia may develop in part as a strategy for coping with the learning demands of childhood, and that the successful application of this strategy molds synesthetic associations to reflect what has been learned. At the outset, we wish to note that this need not be in conflict with genetic or nativist accounts of synesthesia. There may be a particular genetic inheritance or neurological profile that is required for the development of synesthesia, but learning is also a crucial component of this development, as is explicitly acknowledged by researchers with a strongly nativist approach (e.g., Spector and Maurer, 2009). The theory presented here goes further by stating that synesthetic associations are not merely learned, but learned *for strategic purposes*. But strategic purposes do not rule out an equally important role for genetic or neurological factors.

In what follows, we use the standard terminology of "inducer" to refer to the stimuli that trigger synesthetic experiences, and "concurrent" to refer to these experiences themselves, and we generally refer to types of synesthesia by the common formula *inducer–concurrent* (e.g., *time–space* synesthesia refers to time being perceived by the synesthete in a spatial form).

#### **HOW LEARNING INFLUENCES SYNESTHESIA**

Synesthesia typically develops during a period in which children are engaged in the explicit learning of new skills. This often occurs in a formal setting, and involves learning to recognize, discriminate, name, and use the elements of highly structured categories. These may be the letters of the alphabet, the months of the year, the keys of music – the synesthetic inducers themselves. This learning phase is often very slow and deliberate, continuing into the teenage years. As the child learns more about the inducer domain, their synesthetic experiences change. Not only are individual concurrents determined on the basis of this learning, but so too are the relations *between* concurrents. Indeed synesthetic concurrents can be said to *encode* a wide range of information about the inducers, albeit in a highly idiosyncratic manner unique to each synesthete. We justify each of these claims below.

#### **SYNESTHETIC INDUCERS ARE LEARNED CATEGORIES**

A striking feature of almost all cases of synesthesia is that they presuppose complex categorical learning. They involve a consistent, one-to-one mapping of members of the inducer class onto members of the concurrent class, which requires the ability to identify and discriminate the members of the inducer class and, more generally, to understand the nature of that class. A synesthete who sees each *L* as lime green can visually identify that letter by means of its shape. But the synesthete also sees *L as* a letter: a symbol that represents specific phonemes in the context of words, and a member of an ordered sequence, among other things. If an instrument–color synesthete "hears" a piccolo and a flute as two distinct shades of blue, the synesthete must be able to discriminate between the characteristic timbres of the two instruments, but also to classify and recognize them as distinctive timbres. Such understanding does not come easily.

The development of literacy and letter recognition is a notorious example. Even 11 year olds, who have had a minimum of 6 years of literacy training, are affected by letter crowding over larger expanses of text (7.13 threshold keystrokes) than are adults (2.83 keystrokes). In fact, crowding distance is roughly the same for 11 year olds as it is for 8 and 5 year olds (Jeon et al., 2010). Sequential and cyclical systems for marking the passage of time, both natural (e.g., day and night, seasons) and conventional (e.g., days of the week, months) present their own conceptual hurdles. For example, while the child's understanding of sequential order and mistakes in sequential order improves progressively from ages 4 to 11, only at age 10 do children begin to form an integrated understanding of two distinct cyclical orders of events, allowing them to order the months of the year and recognize when standard holidays are located (Friedman, 1977). Similarly, musical categories and relations such as absolute and relative pitch, scales, modes and tunings, and the timbre of specific instruments, all require either extended listening experience and/or explicit musical instruction. These three classes, graphemes, time units, and aspects of music, make up most reported cases of synesthetic inducers in all large-scale studies to date (cf. Day, 2005; Rich et al., 2005; Simner et al., 2006), and, like less common inducers such as swimming styles (Nikolic et al., 2011) are

typically learned only with considerable effort over long periods of time as a result of explicit instruction from teachers or parents.

What about the rarer types of synesthesia that involve "natural" inducers such as odors, sounds, or pains (cf. Day, 2005)? Neonatal behavior such as crying presumably reflects the experiences of the newborn, such as states of hunger, pain, thermal discomfort, or agitation. But despite the presence of such experiences, the perceptual and conceptual skills that undergird developmental synesthesia, that make possible mappings between inducer and concurrent domains, are arguably not present at birth. In fact, what unifies many of these inducers is that (a) they are notoriously difficult to identify and name, (b) their identification and naming is aided by contextual cues and hence these senses are highly multimodal, and (c) discrimination is affected by the naming. To use the example of odor, both odor naming and discrimination have an inverted U-shaped function across age, with young children and the elderly least able to name and discriminate familiar odors (de Wijk and Cain, 1994a,b). Even for young adults, odor naming is very difficult. In blind testing without visual or other cues, adults recognized only 50% of common household odors. The percentage is much lower when less common odors are tested. However, in a phenomenon called "tip of the nose" effect, Jönsson et al. (2005) have shown that providing the name immediately serves to "clarify" the odor for the perceiver. Similarly, when four names are offered in conjunction with the odor sample, if one name matches the sample, correct identification of odor increases from 30% to 80% (again in young participants; de Wijk and Cain, 1994b). Finally, what people smell is strongly influenced by color, and these associations between color and odor are at least partly culturally determined. For instance, Shankar et al. (2010) found that when presented with the same brown colored liquid, 70% of the British participants identified the drink as a cola while none of the Taiwanese participants did so. Instead, 49% of the Taiwanese identified that drink as grape.

Now consider the example of pain–color synesthesia. The few studies that exist suggest that *categories* or *types* of pain are the inducers, e.g., headaches and cramps reliably induce very different concurrents (Coriat, 1913; Dudycha and Dudycha, 1935). But reliably categorizing pain requires learning. At birth, a cold and hungry baby will cry. But electrophysiological recordings suggest that young infants cannot distinguish a soft touch on the heel from a noxious lancing (Fabrizi et al., 2011). Even older children (age 4–11) with a great deal of experience with pain often identify loss of appetite or nausea as pain (Kortesluoma and Nikkonen, 2004). The number and type of pains reported by children is influenced by their own past experience with pain, and by the pains they have observed their parents and siblings experiencing (Harbeck and Peterson, 1992; Franck et al., 2010), and the terms used to describe different types of pain become increasingly more specific through childhood (e.g., Gaffney and Dunne, 1986; Harbeck and Peterson, 1992; Crow, 1997). Moreover, children's judgments of the severity of pain are mediated by cues such as syringes, or their own crying or sweating (Kortesluoma and Nikkonen, 2004). All this evidence suggests that a large part of the ability to categorize the

quality, magnitude, and location of pains is improved and refined throughout childhood.

These examples suggest that the apparent dichotomy between "innate" and "learned" varieties of synesthesia needs closer scrutiny. Certainly, nothing logically precludes innate sensory representations or categorizations. But learning may influence the categorization of *all* synesthetic inducers, not merely the large majority of cases that clearly involve extensive explicit category learning.

In summary, s*ynesthesia normally develops as part of a difficult, formal process of learning to recognize and categorize perceptually and conceptually complex inducers. Even in cases of more unusual inducers, learning – especially perceptually and/or conceptually challenging forms of learning – seems to be at the heart of synesthesia.*

#### **THE SLOW COURSE OF SYNESTHETIC DEVELOPMENT**

Only two published papers directly investigate the development of synesthesia, both derived from the same longitudinal database on grapheme–color associations (Simner et al., 2009a; Simner and Bain, 2013). These show that synesthetic associations coalesce very slowly: 6- to 7-year-old synesthetes have consistent color associations with approximately 35% of letters, which increases to approximately 70% by ages 10–11. Thus there is a period of at least 4 years during which these synesthetic associations are in flux. Unfortunately the published accounts do not tell us about those letters that do not have stable associations with colors. Do they have weak associations, unstable associations, or no associations at all? We anticipate that future child studies will attempt to grapple with this issue.

Interestingly, not a single one of Simner et al. (2009a), Simner and Bain (2013) 615 participants had perfectly stable letter–color associations, a fact that should make us question the common claim of synesthetes that letter colors are fixed "from infancy" or "as long as I can remember" (e.g., Rich et al., 2005). It appears that the color evoked by a letter in adulthood is experienced as the color that *L* has "always been," but these memories are not always accurate.

*Synesthesia often develops slowly, into late childhood.*

#### **LEARNED FIRST-ORDER INFLUENCES ON SYNESTHETIC DEVELOPMENT**

While Simner et al. (2009a) have provided the only studies of childhood synesthetes, the settled associations of adult synesthetes provide insight into the multi-year process of gradual synesthetic stabilization. Just as paleontologists can infer a great deal about long-extinct species by examining their remains, synesthesia researchers can learn about various influences on the development of synesthesia by examining the stable associations that persist into adulthood, making these associations like perceptual/cognitive fossils. We divide these associations into two broad groups: firstorder and second-order mappings. In principle there could be learned or innate influences of both types, but as we illustrate below, in most cases these influences are clearly learned, and the evidence for the innateness of some cases is debatable.

A first-order mapping is one in which single elements in one domain (e.g., the domain of letters) are mapped to single elements in another (e.g., the domain of colors). Cases in which particular letters tend to be associated with the same color categories

for many synesthetes, then, would be first-order mappings. For example, a common finding is that the first letters of common color names tend to be associated with the corresponding color, for example purple is often associated with *P* for English synesthetes (Simner et al., 2005). Similarly, color *words* are generally the color that they refer to (e.g., the English word *blue* is often blue), and the words for objects with prototypical colors are often colored accordingly (e.g., *banana* is yellow) in both English and Japanese (Baron-Cohen et al., 1987; Rich et al., 2005; Asano and Yokosawa, 2012). All of these associations are clearly learned, i.e., based upon semantic or orthographic associations.

More idiosyncratic cases of learned first-order synesthetic associations exist. Some adult grapheme–color synesthetes have colors that derive almost entirely from childhood toys (Hancock, 2006; Witthoft and Winawer, 2006, 2013) or colored letters displayed in kindergarten classrooms (Colizoli et al., 2012). Similar childhood determinants of synesthetic associations have been noted for well over a century (Calkins, 1893). The origins of these associations can be complex and subtle, and can only be unearthed by careful case studies. As an example, Melanie Ahrling gives this account of some of the childhood origins of the colors and form of her synesthetic calendar:

My entire week is composed of elongated rectangles strung together in a slightly irregular fashion – jutting out a little, either at the top or the bottom. The blue Monday and the yellow-red Tuesday are rounded off at the top – similar to an inverted awning. Work-days have more individual characteristics than the weekends. I'm pretty sure that's because as a child, I had a different lesson plan for every day and also varying lesson times in the afternoons. Even then, I never really liked Thursdays which for me are dark-green with black stripes due to years of having to go to children's ballet lessons with an ancient and rather "strict" teacher, which wasn't exactly my idea of fun. My Wednesday however, is always light-blue with little white clouds. This is possibly because throughout my entire childhood we always had Wednesday afternoons off – in comparison to the other weekdays, on which there was always some kind of schedule that had to be adhered to [...]

For the most part, the weekend was filled with enjoyable outings and for as long as I can remember, has been composed of a large block of two rectangles – namely the dark violet-blue Saturday and the violet Sunday. These rectangles were always somewhat bigger than the rest of the weekdays – and still are – to this day. The more rigidly structured the weekend – irrespective of whether the activities are fun or not – the closer these two rectangles are pushed together (Dittmar, 2009, pp. 139–140).

Such recollections are very difficult to confirm, of course, but they are common in self-reports by synesthetes.

Rarer forms of synesthesia, such as the lexical–gustatory variety, also exhibit learned first-order influences on synesthetic concurrents. Thus the flavor of a word is often influenced by shared phonology between the word and the name of the flavor, so *Cincinatti* might taste of *cinammon*, or by semantic associations between the word and the flavor, so *blue* might taste *inky* (Ward and Simner, 2003; Ward et al., 2005). Even without formal instruction in flavor recognition, semantic associations with tastes still develop.

For grapheme–color synesthetes, *O*, *I*, *X*, and *Z* are typically associated with achromatic colors, in particular *O* and *I* with white and*X* and *Z* with black, and*C* is often associated with yellow (Rich et al., 2005; Simner et al., 2005). Spector and Maurer (2008, 2009, 2011) investigated these associations with two-alternative forcedchoice tasks in which participants searched for plastic shapes in containers that were divided into two differently colored halves. They found that non-synesthetic adults, children, and pre-literate toddlers reliably chose to look into the white half of containers when searching for *O* or *I* shapes, the black half of containers when searching for *X* or *Z* shapes, and the yellow half of containers when searching for *C* shapes. These age groups differed, however, in other cases: older children and adults looked into the blue half of containers to find *B* shapes, the yellow half of containers to find *Y* shapes, and the green half of containers to find *G* shapes, but toddlers did not. These results demonstrate that certain shape– color associations in toddlers are not the result of learned semantic associations between letter names and color names. It is controversial, however, whether stronger conclusions about innateness are warranted. First, several environmental sources of these associations have been proposed (cf. Spector and Maurer, 2011). Second, the relationship between these shape–color associations and grapheme–color synesthesia is unclear. Graphemes are not shapes *per se*, but simply the smallest contrastive unit in an orthography that is capable of distinguishing meaning. Thus the same letter can take on many different shapes and a grapheme–color synesthete will generally see all these shapes as having the same color, e.g., *A* and *a* would be seen as the same shade of crimson (for a rare report of an exception to this rule, see Ramachandran and Hubbard, 2001a), and identical shapes that are contextually determined to be different graphemes will be perceived as different colors (Dixon et al., 2006). Furthermore, for adult synesthetes graphemes induce highly specific colors (approximately as specific as color memory Arnold et al., 2012), not simply color categories or any nearby colors irrespective of color category boundaries. Thus further experimentation is required to resolve several questions. What is the precise nature of the toddler associations – what aspect of shape is associated with what aspect of color? Are these associations learned or innate? And are they the basis of the later associations between graphemes and highly specific colors in adult synesthetes?

It is important to note that despite all these first-order influences, any two synesthetes' sets of inducer–concurrent pairings are likely to appear entirely unrelated to each other. Even the most powerful first-order effects, such as *O* being white for grapheme– color synesthetes, have many exceptions, and most of these effects are only apparent after careful statistical analyses of large populations. However, the vast majority of synesthetes we have had the opportunity to study appear to be influenced by at least some of these factors.

*The synesthetic concurrent that is associated with a particular inducer is often determined by what the synesthete has learned about that inducer or experiences they have had with the inducer*.

#### **SECOND-ORDER INFLUENCES ON SYNESTHETIC DEVELOPMENT**

Second-order mappings can be thought of as "relations between relations." Unlike first-order mappings, in which single elements in one domain are mapped to single elements in another domain (e.g., the letter *A* is mapped to the color red), in second-order mappings, patterns, or relations within one domain are mapped to patterns or relations within another domain. On reflection this is not surprising, given that human perception often deals with relationships rather than absolute stimulus values: for instance we can identify a tune such as Happy Birthday (a pattern of relationships between notes) better than we can individual notes.

A number of second-order mappings influence synesthetic concurrents. For instance, letters that have similar *shapes* (e.g., *E* and *F*) tend to have similar *colors* among grapheme–color synesthetes whose native language is English (Brang et al., 2011b;Watson et al., 2012a), German (Jürgens and Nikolic, 2012), or Japanese (Asano and Yokosawa, 2013). In these cases, similarity relationships within the letter–shape domain are mapped onto similarity relationships within the color domain. Second-order mappings such as these can be entirely independent of first-order mappings, and thus would not show up using first-order analyses. Thus *E* might have completely different colors for synesthetes John and Jane, but if John's color for *E* is similar to his color for *F*, and Jane's color for *E* is similar to her color for *F*, then the two have the same second-order mapping of shape to color.

Similarity in terms of shape is not the only type of letter similarity that affects synesthetic color. The relative frequency of letters also matters: there is a tendency for letters and numbers that are more frequently seen in print to be associated with brighter colors (Beeli et al., 2007; Cohen Kadosh et al., 2007; Smilek et al., 2007; Simner and Ward, 2008; Watson et al., 2012a), with more saturated colors (Beeli et al., 2007), and with colors whose names are more commonly spoken or written (Rich et al., 2005; Simner et al., 2005). Thus there is a second-order mapping between similarity in terms of letter/digit frequency and similarity in terms of luminance, saturation, and color word frequency.

The phonological similarity of letters also affects their synesthetic colors, although this may be a language-specific effect. Japanese *hiragana* and *katakana* characters with the same pronunciations tend to have the same colors, despite large differences in shape (Asano and Yokosawa, 2011), and hiragana characters with similar pronunciations tend to have similar colors (Asano and Yokosawa, 2013). However phonology may not be able to influence synesthetic color in opaque orthographies, where letters have multiple pronunciations. In English, for example, similarity between the sounds of letter names does not correlate with synesthetic color similarity (Watson et al., 2012a).

Another important form of letter similarity is in terms of alphabetical order, which has a somewhat curious relationship with synesthetic color: letters that are earlier in an alphabet tend to have less similar colors than later letters, at least among native speakers of English (Eagleman, 2010; Watson et al., 2012a) and Japanese (Asano and Yokosawa, 2013). The original explanation of this finding treats alphabetical order as a kind of proxy for the learning order of letters, as letters are generally learned in roughly alphabetical order (Justice et al., 2006). The idea is that as synesthetic children begin learning letters, they assign each letter a distinctive color, but as they learn more and more letters they run out of colors and are forced to use colors that are similar to those they had already assigned to previously learned letters (Eagleman, 2010; Watson et al., 2012a).

This interpretation, however, is hard to reconcile with the lengthy developmental trajectory of synesthesia. By age 7, the majority of children will have learned all their letters, but the synesthetes among them will have consistent color associations for only a third of these letters (Simner et al., 2009a), and will not develop color associations for many of them until their teenage years. It seems unlikely that these associations develop in alphabetical order, given that all the letters are wellknown by this point. Thus most letter–color associations are not developed in anything like the order of learning of letters, which casts doubt on the theory that synesthetic alphabetic ordinality–color effects come from the order of acquisition of letters. An alternative possibility is that ordinal position is itself an important aspect of how alphabetic letters are conceptualized and encoded in the brain, and that any such important aspects will inevitably have an impact on synesthetic color (cf. Asano and Yokosawa, 2013).

Music–color synesthesia also displays second-order influences, although it is less clear that these influences are due to learning. Higher pitches tend to be associated with brighter colors (Marks, 1975), and quartertones tend to be associated with colors that are closer to the midpoint of the colors of the two adjacent semitones (Head, 2006). This pitch–brightness correspondence is not a unique feature of synesthesia: it is ubiquitous among non-synesthetic humans and even chimpanzees (Ludwig et al., 2011).

These second-order effects are independent of each other, at least in the case of grapheme–color synesthesia. Thus synesthetes who display a strong ordinality–color effect are no more or less likely to display a strong shape–color effect, and so forth (Watson et al., 2012a). And, as with the first-order effects, any synesthete might show no signs of a given second-order effect: these are statistical tendencies, not firm rules. Nevertheless, they can be powerful indeed, as shown by the fact that fully 79% of the variance in synesthetic color difference for Japanese hiragana is explained by a model that includes ordinality, shape, phonological, and familiarity differences (Asano and Yokosawa, 2013).

Not all second-order effects need to be learned, and several have been interpreted as arising from general developmental processes independent of experience (Maurer et al., 2012). These include mappings between visually perceived angle, height, lightness, size and auditory pitch, and sounds and shapes. One might also argue that, e.g., shape–color and sound–color similarity mappings in grapheme–color synesthesia are due to an innate tendency to map similarity to similarity. However the identification and discrimination of the individual elements in many of these mappings will require extensive learning as well (the difficulties in learning to reliably discriminate letter shapes, for instance, were discussed in the previous section), and learned contingencies could also account for some of these mappings (as is noted by Maurer et al., 2012). Furthermore, there is strong opposition from Deroy and Spence to the notion that *any* of these associations are unlearned, and several methods by which they could arise from environmental learning have been proposed (Spence and Deroy, 2012; Deroy and Spence, 2013a,b). As with the discussion of first-order relations, we do not intend to deny the possibility of unlearned influences on second-order relations, but we do suggest that

learning will be an important influence on most, if not all, of them.

Finally, we note a rare example of how explicitly understood similarity relations can be used to create synesthetic mappings. In order to be able to sing in foreign languages, a problem for opera singers, Jasmin Sinha uses a synesthetic version of the international phonetic alphabet (IPA) chart of vowel phonemes. She explains how she sings vowels correctly:

When I need to identify a vowel accurately in order to sing it, I always make mental use of the (IPA) vowel chart that I first encountered as a tool while studying linguistics at university. I can assign the vowel that I need to sing to a precise location on the chart and it hardly ever coincides exactly with one of the spots that are marked. I almost always need to move the position of the desired sound on the chart a little bit.

Synesthesia does not come into it until the next stage, when I start to make conceptual use of what I learned in my linguistic studies [...] I see the layout of a vowel chart on my internal monitor, which is set up about 20–30 centimeters in front of me like a cinema screen. This screen is not flat; it curves in a wide arc in front of my face, but does not encircle my head. It looks like a large, under-exposed and grainy old black and white photograph, in which nothing can be made out except the grainy texture; the edges are rather faded. The basic color tends towards what is almost a very dark sepia; it is not simply black.

With a conscious act of will I project a horizontal vowel chart onto this screen. The chart's bottom line is quite close to my chin and it then turns away from me with a slight lean to the left (this means that the open "a" is closest to my chin). In addition to the fact that my subjective vowel chart does not correspond exactly with the official one, I have given it an extra personal and synesthetic dimension, resulting in a threedimensional structure. The non-synesthetic grid, which is based on the official vowel chart, forms the upper part; it "floats" above a colored layer which depicts my vowel colorings (Dittmar, 2009, p. 199).

In summary, the *relations between synesthetic concurrents often preserve many of the important learned relations between their inducers*.

#### **CAN SYNESTHESIA BE LEARNED BY ADULTS?**

A number of studies have tried training adult non-synesthetes to make synesthetic–style associations. These studies have all employed different training paradigms, such as trial-and-error learning in which participants choose from among several colored patches when presented with a black grapheme and are given feedback on each choice (Brang et al., 2011a), having participants read novels with colored letters at their own pace (Colizoli et al., 2012), speeded visual search tasks with colored letters (Kusnir and Thut, 2012), or even post-hypnotic instructions to associate particular numbers and colors (Cohen Kadosh et al., 2009). Training times range from 15 minutes or less (Brang et al., 2011b) to 20 days (MacLeod and Dunbar, 1988), although most are well under an hour. This heterogeneity makes cross-study comparisons difficult, but there are several points worth considering.

These studies show that non-synesthetes can behave like synesthetes on the "synesthetic Stroop task." In this task, inducers are presented in colors that are either congruent or incongruent with their associations and the subject must name the color as quickly as possible. Response times are slower when the color is incongruent with their association (Mills et al., 1999; Dixon et al., 2000). Several training studies find a similar effect

among non-synesthetes with trained associations (Elias et al., 2003; Colizoli et al., 2012; experiments 2 and 3 in MacLeod and Dunbar, 1988; Meier and Rothen, 2009), although others do not (Kusnir and Thut, 2012; experiment 1 in MacLeod and Dunbar, 1988), which may be due to insufficient training time (cf. MacLeod and Dunbar, 1988).

Phenomenological reports from participants in these studies are particularly interesting, although only one study collected these in a formal manner (Colizoli et al., 2012), and several give no account of phenomenology at all (Calkins, 1894; Nunn et al., 2002; Elias et al., 2003; Brang et al., 2011b; Brang et al., 2013). Participants in 3 studies reported not having color experiences (Kelly, 1934; Meier and Rothen, 2009; Kusnir and Thut, 2012), but other studies have more promising results. Several participants in the colored novel-reading study reported that they began experiencing colors when thinking about certain letters, and the degree to which they endorsed having such experiences was correlated with the strength of their synaesthetic Stroop effects (Colizoli et al., 2012). One participant in a shape-color training study reported that "the shapes began to take on the color of the names assigned them even on training days, when they appeared in white" (MacLeod and Dunbar, 1988). Participants in the post-hypnotic suggestion study reported that numbers on license plates or street signs took on their associated colors (Cohen Kadosh et al., 2009). Finally, participants in a tone-color training study reported seeing a color when its corresponding tone was played, even if the actual visual stimulus was faintly colored with a different hue (Howells, 1944). Importantly, then, subjects in one training paradigm (Colizoli et al., 2012) report experiences that are not unlike reports made by "associator" synaesthetes, who do not experience their concurrents located in external space (Dixon et al., 2004), particularly "know-associators," who do not experience their concurrents in a strong perceptual manner at all (Ward et al., 2007), while subjects in others (Howells, 1944; MacLeod and Dunbar, 1988; Cohen Kadosh et al., 2009) report experiences that resemble reports of "projector" synaesthesia, in which participants experience concurrents in the external world (Dixon et al., 2004), specifically the "surface-projector" variety, in which these concurrents appear on the inducers themselves (Ward et al., 2007). It is also interesting to note that three studies which had participants who reported synaesthesia-like experiences were among the longest studies (Howells, 1944; MacLeod and Dunbar, 1988; Colizoli et al., 2012), and one showed that an identical training regimen used for a shorter length of time did not lead to such reports (MacLeod and Dunbar, 1988). No doubt there are important differences between the experiences of participants in training studies and long-term synaesthetes, but the weight of the evidence from selfreports certainly seems to suggest that there are equally important similarities.

Phenomenal experiences aside, trained participants in these studies differ from developmental synesthetes. Participants in the novel-reading study had poor recall of their letter–color associations, remembering less than half of their associations 6 months after training had stopped, whereas synesthetes typically have close to perfect consistency in their reported associations over much longer periods of time (Colizoli et al., 2012). Participants who had been trained with letter–color associations and subsequently learned conditioned responses to colors did not transfer these responses to their associated letters, unlike synesthetes given the conditioning task (Meier and Rothen, 2009). Nor did participants trained with word–color associations show increased activation in visual cortex when listening to words, unlike matched synesthetes (Nunn et al., 2002). However given that the synesthetes in such studies have had decades of experience with their associations, the short time spent in training (respectively, long enough to read a single novel, 70 min, and possibly less than 10 min) is hardly sufficient to draw any strong conclusions (as is noted by Meier and Rothen, 2009). One neurophysiological study did test an individual with much more training: a cross-stitcher with 8 years of experience viewed numbers that she associated with thread colors (Elias et al., 2003). Unlike a grapheme–color synesthete performing the same task, she showed no activation of visual cortex. Note, however, that this is a single null result from a single subject. Given that only two studies have compared the neurophysiological correlates of trained and synesthetic associations (Nunn et al., 2002; Elias et al., 2003), and given that neurophysiological results in developmental synesthetes are notoriously hard to replicate (see Rouw et al., 2011 for a review), a pair of null results using different training paradigms and different types of synesthesia is impossible to interpret.

In summary, *non-synesthetic participants in a number of training studies report experiences that sound quite similar to genuine synesthetic experiences. There are important differences between the effects of short-term training in adults and the long-term associations of synesthetes, but these are not enough to conclude that the two are qualitatively distinct.*

#### **HOW SYNESTHESIA INFLUENCES LEARNING**

Above we reviewed the studies that have examined how learning affects synesthetic experience. Influences in the other direction – from synesthetic experience to its use in learning – have received less attention. In this literature researchers have focused primarily on enhanced memory. There have been very few studies on the complex forms of learning that synesthetes commonly report their synesthesia helps with, such as learning to speak foreign languages, how to score music, to hear musical intervals and musical key changes, and to understand correct musical phrasing, or even to hear the phonemes of vowelsfor both native andforeign languages. "Pure"memorization is clearly necessary but not sufficient for such tasks, meaning that there is a lack of evidence about potentially important aspects of synesthesia.

In a nutshell, the studies summarized below suggest that synesthesia can enable both implicit and explicit forms of learning. That is, synesthetes can exploit their concurrents as a means of obtaining and retaining information about their inducers. In some cases they do this deliberately, even to the extent of planning or training themselves to use their concurrents in this manner, while in other cases they learn spontaneously and without apparent awareness.

#### **THE DIFFICULTIES OF STUDYING SYNESTHETIC LEARNING**

There are a number of reasons why this area of research is underdeveloped. In part, the problem lies in delineating a number of specific claims and finding the appropriate experimental methods to test them. Here we list some of the complications facing experimenters in this field.

First, we need to ask whether synesthesia helps synesthetes to learn and/or organize their thoughts or whether it merely *seems* to do so. It is possible that despite their rich experiences, synesthetes use the same cognitive resources as non-synesthetes to remember their appointments throughout the week, sing the proper vowels and so on. Is their phenomenology actually connected to how synesthetes learn? Seron et al. (1992), for example, report that they were unable to determine whether their participants'numberforms are actually used in calculation.

Second, synesthesia might provide an *alternative* way to master a skill, but not necessarily a *superior* way. Certainly the superiority claimed by many synesthetes could result from a failure to imagine how non-synesthetes get about the world without these experiences. As Melanie Ahrling remarked: "I don't understand how anyone can orientate themselves in time without such schemata in their heads, but obviously it does work and for many people it functions just as well for them as it does for me – except without the colors and little boxes." (Dittmar, 2009, p. 147). In this case, studies that merely look for advantages on certain tasks might be missing important differences.

Third, synesthetic styles of learning might even prove a hindrance. An example of this comes from a synesthete who reports that when she tried to learn the piano, she discovered that she had three different and inconsistent sets of colors: one for each of her fingers, one for each of the musical pitches, and another for the letter names of the musical notes. These inconsistencies proved insurmountable, and so she gave up trying to learn the piano and switched to the Theremin (Pautzke, 2010)! A recent study has also shown that grapheme–color synesthetes are impaired when trying to memorize randomly chosen associations between colors and graphemes for which they do not consciously experience synesthetic colors (Brang et al., 2013). In general, any advantages might be counterbalanced by limitations.

Finally, synesthesia often involves complex, multimodal experiences. For example, a grapheme–color synesthete might experience the letter *P* as a specific stable color. But *P* might also have any number of other properties (Eagleman and Goodale, 2009): it could emit light or glow, be composed of a mix of colors, have a certain level of specular reflectance (shiny or dull), a texture (silk or sandpaper), a personality (boring or overbearing), or a taste (salty or sour). Any or all of these properties might be relevant to the role that *P* plays in the individual's system of learning.

A recent documentary illustrates this complexity nicely (Kirschner and Söffig, 2012). In it, a number of synesthetic members of a boys choir discuss how they use their synesthesia to"code" a heard song in order to reproduce it. For example, one boy sees each note as a colored square; volume affects the size of the square and its color represents pitch. Another boy represents the song as a series of colored dots that form a line stretching out to the left of the child; each note within a phrase receives the same color, and pitch is represented by a line of dots that dips and rises. A third represents the same song with elaborately colored, personified, hopping numbers, shapes and stick persons.

What the boys are learning, namely rote-singing, is a complex set of skills that allow one to reproduce long passages of music from a single hearing. They must learn to hear the music, to discriminate changes in pitch and discern the melodic contour; to commit the melodic phrase to memory; and finally to reproduce the melody, matching the voice to each remembered note while monitoring and correcting vocal errors. Like chess masters who perceive structure and relations between pieces in a few glances (cf. Reingold and Charness, 2005), skilled rote singers have learned to perceive musical relations rather than collections of notes. This is true even for musicians with perfect pitch and for musical savants (Halpern and Bower, 1982; Sloboda et al., 1985; Young and Nettelbeck, 1995; Miller, 1999). Children who are explicitly taught pitch relations and the melodic contours of tonal music through explicit explanations of musical concepts are able to learn melodies more quickly, understand melodic contours better, make fewer mistakes in memory, and sing more accurately, results that are in line with adult performance (Petzold, 1963; Apfelstadt, 1984; Oura and Hatano, 1988; Hedden, 2012).

One can imagine several ways in which each of the boys' synesthesias might help with any of these skills, but confirming that this is what actually happens would be difficult indeed. Recall the boy who experienced melody as an undulating line of colored notes. An obvious interpretation is that he represents pitch contour spatially and note identity with color. Even so, his perception of pitch contour might be no better than average. He may have substituted his synesthetic representations of relational pitch for whatever representational methods are used by non-synesthetes, in much the same way that children who are taught alternative multimodal forms of representation may use those types of representation. Then again, despite his own reports, his synesthetic experiences may play no role in his memory of pitch sequences. Finally, even if his memory for pitch contour is exemplary, there is no guarantee that his singing will be more accurate than that of other choir members. Vocal production of a song requires more than accurate memory, such that relational encoding fails to enhance vocal matching. In this particular case, the choirmaster reports that the synesthetic boys are not especially skilled in comparison to the other choir members (Kirschner, personal communication).

For many reasons, then, understanding and testing how synesthesia affects learning is truly difficult, and will often require a deep understanding of the specifics of each individual synesthete's experiences in order to develop appropriate experiments (cf. Smilek and Dixon, 2002). Despite these challenges, there has been some success in verifying that synesthesia can and does affect certain forms of learning, and we now turn to these cases.

#### **IS THERE A SYNESTHETIC ADVANTAGE FOR MEMORY?**

Probably the single most common anecdotally reported benefit of synesthesia is that it helps with memory. This example is typical:


Because the name August is a violet-dark blue and the name "Berthold" is a yellow–brown, I have never in my entire life got these birthdays muddled up – as my number 8 is dark-blue and my 3 is yellow. ....In this instance, the colors of the names and the colors of the birthday– dates match up so well that they serve as an excellent memory aid (Dittmar, 2009, p. 140).

Grapheme–color synesthetes often describe how their colors assist with remembering names, telephone numbers, or the spellings of words (cf. Chapter 2 inCytowic, 2002), and they generally report above-average memory abilities (Yaro and Ward, 2007). Several recent studies have formally investigated the relationship between synesthesia and memory, using both case and group studies. Readers interested in a more detailed account of this research should consult the review by Rothen et al. (2012).

Single case studies usually focus on synesthetes with extraordinary memory abilities. These include synesthetes who explicitly claim to have exploited their synesthesia to aid them in memorizing pi to over 20,000 decimal places (Bor et al.,2007), remembering long lists of names for many weeks (Mills et al., 2006), perfectly recalling a list of random words on a surprise test 20 years after initial exposure (Luria, 1968), or perfectly recalling several 50-digit matrices several months after studying them for a few minutes (Smilek et al., 2002). In the latter case, researchers verified the synesthete's claim that she was exploiting her synesthetic colors on this task by presenting her with matrices of numbers that were colored incongruently with her concurrents, which caused her performance to plummet well below the level of non-synesthetes (Smilek et al., 2002).

Group studies, on the other hand, have focused mainly on grapheme–color synesthesia, with one study examining time– space synesthetes (Simner et al., 2009b). In all cases, they have shown either no general memory differences between synesthetes and non-synesthetes, or synesthetic advantages that are impressive, but not near savant levels (cf. Rothen and Meier, 2010a).

What are these advantages? Synesthetes tend to have superior memories for the specific stimuli of their concurrent domain(s), e.g., grapheme–color synesthetes have advantages on various tests of color memory (Cohen's *d* or Glass' ranging from 0.54 to 1.20; Yaro and Ward, 2007; Rothen and Meier, 2010a; Pritchard et al., 2013; Terhune et al., 2013), and calendar–form synesthetes have visual short term memory advantages (Cohen's *d* = 0.35, Simner et al., 2009b). Furthermore, they often show memory advantages for stimuli from their inducer domain, thus grapheme–color synesthetes have advantages for various memory tasks involving lists of words (Cohen's *d* ranging from 0.75 to 1.38; Yaro and Ward, 2007; Rothen and Meier, 2010a; Gross et al., 2011; Radvansky et al., 2011). They are also better at implicitly learning artificial grammars, but only if these grammars are composed of the letters that trigger their synesthesia (Cohen's *d* = 0.62; Rothen et al., 2013). Calendar–form synesthetes have much better autobiographical and somewhat better historical memories (Cohen's *d* of 1.95 and 0.87, respectively; Simner et al., 2009b).

However synesthetes show no advantages for some other memory tasks involving their inducers. Grapheme–color synesthetes do not appear to have a group advantage for remembering matrices of numbers (Yaro and Ward, 2007; Green and Goswami, 2008; Rothen and Meier, 2009), nor for retaining graphemes in working memory (Rothen and Meier, 2010a; Gross et al., 2011; Terhune et al., 2013). Furthermore they have advantages in the reproduction and recognition of simple visual figures that are not associated with their synesthetic inducers or concurrents at all (Rothen and Meier, 2010a). Thus not all synesthetic memory advantages can be tied to synesthetic experiences.

#### **IS THERE A SYNESTHETIC STYLE OF MEMORY?**

Clearly, the research on synesthesia and memory has ambiguous results. We suggest that this is because this is a relatively new area of research and the appropriate experimental methods have not yet been determined. Most studies have concerned themselves primarily with determining whether synesthesia is associated with memory advantages, which is an interesting question in its own right, but may be irrelevant to determining whether synesthesia is exploited for memory. As discussed above, the question is not whether synesthetes perform *better* than non-synesthetes on memory tasks, but whether they perform *differently*.

What is known about such differences comes either from selfreports or from a single experimental technique that has been employed in a handful of studies: using incongruently colored stimuli in an attempt to impair synesthetes' performance (Smilek et al., 2002; Yaro and Ward, 2007; Green and Goswami, 2008; Rothen and Meier, 2009; Radvansky et al., 2011). At present two of these studies have found no evidence of interference from incongruent colors among groups of synesthetes (Yaro and Ward, 2007; Rothen and Meier, 2009), while two group studies and one case study have found interference (Smilek et al., 2002; Green and Goswami, 2008; Radvansky et al., 2011).

One possible explanation for these inconsistencies is the choice of tasks. The stimuli used in most group studies are not ones that synesthetes have reported using their concurrents to help remember. Names have been used in one only study, which did find a strong benefit for the (single) synesthetic participant (Mills et al., 2006), and to our knowledge no one has formally tested synesthetic recall of phone numbers. Group studies where adult synesthetes were tasked with remembering large matrices of numbers showed no synesthetic advantage (Yaro and Ward, 2007; Rothen and Meier, 2009), but the synesthetes in these studies had not previously claimed to have unusual memories for such matrices. On the other hand, in every case where synesthetes have been tested *on the class of stimuli for which they actually claim to use synesthesia as a memory aid*, they have shown advantages (Luria, 1968; Smilek et al., 2002; Mills et al., 2006; Bor et al., 2007; Yaro and Ward, 2007; Simner et al., 2009b; Rothen and Meier, 2010a; Gross et al., 2011; Radvansky et al., 2011). Note that in the two cases where such advantages have been demonstrated and incongruently colored stimuli were employed to interfere with synesthetic experiences, synesthetic memory advantages were mitigated (Radvansky et al., 2011) or eliminated entirely (Smilek et al., 2002).

Clearly more research is necessary, hopefully involving novel ways of testing for specifically synesthetic styles of performance on memory tasks. Nevertheless, two key points have now been well established. *First, synesthetes tend to have memory advantages* *of about 0.5–1 standard deviations over non-synesthetes for specific types of stimuli, particularly those from their inducer and concurrent domains. Second, in at least some cases, they can exploit their concurrents in the encoding or retrieval process, and this use of synesthesia as a deliberate mnemonic can be powerful indeed.*

#### **SYNESTHESIA ENABLES NON-DECLARATIVE LEARNING**

With only one exception (Rothen et al., 2013), the memory studies from the previous sections all tested explicit memory, and did so by asking participants to consciously study stimuli for later recall. One study examined whether synesthesia can also enable more implicit forms of learning, specifically classical conditioning (Meier and Rothen, 2007). Participants passively viewed a stream of slides that were either pure color patches, the colors of which changed from trial to trial, or were white with graphemes on them. During a conditioning phase, slides of one color, e.g., blue, were presented simultaneously with a startling sound. Synesthetes began displaying heightened skin conductance responses to both blue slides and white slides with graphemes that triggered blue concurrents, but not to other colors or graphemes, despite the fact that the graphemes had never been presented together with the sound. Thus conditioned responses to colors transferred to graphemes, indicating an unusual form of non-declarative learning. *Synesthetic associations, then, can be the basis of implicit learning about environmental regularities.*

#### **SYNESTHESIA ENABLES EXPLICIT CATEGORY LEARNING**

Only one study to date has sought to verify that synesthesia can be exploited for more complex learning (Watson et al., 2012b). In this study, grapheme–color synesthetes performed a difficult category learning task where stimuli (pairs of letters) were custom-chosen for each synesthete such that the simplest rules that defined each category were based on their synesthetic colors, e.g., "a pair of red and green letters belongs in Category 1." Crucially, each synesthete's stimulus set contained several letters with highly similar colors. If B and R were both red, they could be used interchangeably. Synesthetes were very successful at learning this category structure, and results show that they exploited their colors to do so.

Synesthetes viewing black letters reported that they learned to consciously use their colors to categorize stimuli, though they had not been instructed to do so, and their pattern of results was consistent with this report. Specifically, they learned to categorize training stimuli accurately and after training they could correctly generalize their performance to novel stimuli that followed the same color rules. Furthermore, on a memory task in which they were asked if they had seen a stimulus before, they were unable to correctly reject novel stimuli that followed the same color rules, indicating that they had been attending to the synesthetic colors of letters, rather than to the specific letter identities. On all three tasks (training, transfer, and memory) their performance was similar to that of non-synesthetes viewing colored letters, and unlike that of non-synesthetes viewing black letters. The non-synesthetes viewing black letters were less accurate at categorizing training stimuli, unable to categorize novel stimuli correctly, and much

*more* accurate at correctly rejecting novel stimuli that followed the color rules. These differences all make sense, given that they had no knowledge of the letters' colors and could only attend to letter identity: their initial learning was poor because they could not utilize the simpler color rules, they could not use color rules to transfer to novel stimuli, and they could not be misled by memory foils that were similarly colored to stimuli they had seen before.

Again, more research is clearly needed into the use of synesthesia for more complex learning tasks. Nevertheless this single study demonstrates quite clearly that *synesthetes can consciously use their concurrents to make novel and difficult abstractions and categorizations.*

## **A LEARNING HYPOTHESIS OF SYNESTHETIC DEVELOPMENT**

This paper has so far consisted of a review of research on how learning influences synesthesia and how synesthesia influences learning. We have tried to present results as objectively as possible. Now, however, we switch to a completely speculative mode, and present a novel theory of synesthetic development that brings both streams of research together. As with most research on synesthesia, this theory is most fully developed in regards to grapheme–color synesthesia, but it is meant to apply to most known varieties or synesthesia, and certainly to all those varieties with explicitly learned inducers (which, as we noted previously, constitute the vast majority of known cases). We outline its relationship to other theories of synesthetic development and try to anticipate objections. An earlier version of this theory has already been published (Watson et al., 2010), and we have outlined aspects of it in discussions of our experimental work (Watson et al., 2012a,b), however this is the most complete and up-to-date presentation of these ideas.

#### **SYNESTHESIA DEVELOPS BECAUSE IT IS USEFUL FOR LEARNING**

The theory states that synesthesia develops, at least in part, as a response to the challenges involved in learning to recognize, discriminate, and understand the relations between the members of the inducer class. That is, grapheme–color synesthesia develops because it helps the child learn various things about letters, time– space synesthesia develops as a means of learning about units of time, and so on. Similar claims were advanced well over a century ago (Galton, 1881; Calkins, 1893), but this line of thinking has not been prominent in modern research (with some exceptions, e.g., Seron et al., 1992).

Almost all synesthetic inducers, as we saw above, are category structures that are learned with much difficulty over a lengthy period of time, including rare inducers such as olfaction, taste, and audition. How might synesthesia help with this learning? We have reviewed evidence that synesthetic associations can be exploited for a variety of learning purposes. Most of this research focuses on synesthesia as a conscious memory aid or mnemonic device, and one of the initial problems that faces any child learning to use letters (or other common categories of inducers) is simply to learn to identify which letter is which. This is a far more difficult problem than is typically acknowledged, and the potential utility of a memory aid should not be underestimated. Thus for some grapheme–color synesthetes, synesthesia might arise when they are learning their letters.

However children learn a great deal more about letters than how to recognize and identify them. And while there is a paucity of research on synesthesia's utility for other forms of learning, we have seen that synesthetic associations enable the unconscious learning of environmental regularities (Meier and Rothen, 2007; Rothen et al., 2013), and can also be consciously exploited on difficult rule-based category learning tasks (Watson et al., 2012b). In both cases, the extent and limitations of these abilities have not yet been established, but we suggest that any ability to pick up on statistical regularities of printed letters or to learn complex rules for combining letters could be highly useful at many stages during the development of literacy. More generally, such abilities could be useful when a child is faced with any of the challenges involved in learning any of the categories of synesthetic inducers.

Synesthesia in adults is generally accepted to be automatic, although there is debate over this claim (cf. Mattingley, 2009; Price and Mattingley, 2013). If it begins as a strategy, it seems unlikely that it would be straightforwardly automatic in its early stages, but with extensive rehearsal synesthetic associations could become more and more automatic as time went on, as has been demonstrated in non-synesthetes trained with synesthesia-like associations for 20 hours over the course of a month (MacLeod and Dunbar, 1988). Of course all the synesthetes tested for automaticity using Stroop or other tasks in the laboratory have had a decade or more to rehearse their associations, which would conceal any slower, strategic origins.

#### **MODIFYING CONCURRENTS AS A LEARNING AID**

The potential utility of synesthesia as a learning strategy is magnified by the fact that synesthetic concurrents encode information about their inducers. As we saw, concurrents reflect both the first-order properties of individual inducers (e.g., the semantic associations of particular letters) and the second-order relationships between inducers (e.g., the relative positions of letters in the alphabet). Thus not only can synesthetic concurrents be a basis of learning about inducers, but they are also "chosen" and even modified on the basis of what has been learned. Given the multi-year period in which synesthetic associations are in flux, there is the potential for a lengthy period of reciprocal interactions between learning about inducers and tweaking the particular associations triggered by these inducers. This type of reciprocal relationship may enable further learning.

We have seen, however, that a wide range of inducer properties are mapped to concurrents. It is not possible for all these properties to determine the concurrents associated with each inducer, since they are not all consistent. For instance, *B* and *D* have highly similar shapes (particularly in their lowercase forms), and are also both early letters in the alphabet. Thus according to a mapping of shape similarity to color similarity, they ought to be quite close to each other, but according to a mapping in which letters earlier in the alphabet have *less* similar colors, they ought to be far apart. The semantic associations with these letters might provide a further reason for them to be far apart, as *B* might, for example, be associated with blue as its first letter, but *D* might be brown as a result of being associated with "dog," which are stereotypically brown. Alternatively, such semantic associations might bring them

close together, as *B* could instead be associated with brown as its first letter. Add into this mix their high degree of phonological similarity, differences in their frequency of usage, and the idiosyncratic personal experiences of the individual synesthete with these letters. It is obviously not possible for any two colors for *B* and *D* to simultaneously satisfy all these constraints.

What determines which constraint gets satisfied in any particular case? Asano and Yokosawa (2013) suggest that the feature making the greatest contribution to differentiating the letter from others will be the one that is most likely to have the largest influence on its color. For Japanese hiragana and katakana scripts, which are almost perfectly *orthographically transparent* (each character represents only one phoneme, and each phoneme is represented by only one character), one might expect that phonology would be a particularly important determinant of synesthetic color, while in an orthographically opaque language such as English, phonology would have far less utility.

This analysis resembles the application of optimal integration theory to multisensory perception (Ernst and Banks, 2002). This theory proposes that participants, when required to combine sensory signals from multiple sources, tend to weight the signals according to their relative reliability. Specifically, the weight of a sensory cue in a multisensory decision is directly proportional to the cue's reliability. For example, in a study by Helbig and Ernst (2007), participants made shape judgments of elliptical objects presented simultaneously by touch and vision, but varied slightly in their shape in each modality. The results showed that when noise in the visual signal was low, decisions were dominated by vision, but as the visual noise increased, decisions were increasingly influenced by the signals received through touch. Consistent with this theory, we argue here that the utility of a synesthetic color for discriminating among letters is likely one of the factors determining its selection, but there could be many other such factors. Stated more generally, *synesthetic concurrents are likely determined by whatever aspect of the inducers is most relevant to the resolution of the learning challenge confronting the child.*

#### **WHAT WOULD MAKE SOMEONE MORE LIKELY TO EMPLOY SYNESTHETIC ASSOCIATIONS AS A LEARNING STRATEGY?**

Recently it has become clear that synesthetes, as a group, have a number of unusual cognitive, perceptual, and personality traits. As noted previously, synesthetes have unusually strong perceptual and cognitive abilities related to their inducer and concurrent domains (e.g.,Yaro and Ward, 2007; Simner et al., 2009b), but there are several other elements of this profile, including a strong connection to creativity and artistic expression (Dailey et al., 1997; Rich et al., 2005; Ward et al., 2008; Rothen and Meier, 2010b), skill at making associations between typically disparate elements (Ward et al., 2008), high rates of positive schizotypal experiences (Banissy et al., 2012), stronger verbal and vivid imagery cognitive styles (Meier and Rothen, 2013), enhanced visual imagery abilities (Barnett and Newell, 2008; Spiller and Jansari, 2008; Price, 2009), and greater openness to experience and tendency to fantasize (Banissy et al., 2013).

Consider a child who is open to novel experiences, highly skilled with visual imagery, unusually creative, skilled at making strange associations, prone to fantasizing, and possessed of a highly developed ability to discriminate and remember colors. Such a child could well be far more likely to employ an unusual visual-imagery-based strategy such as associating letters with colors when faced with the multiple challenges involved in learning to read. We suggest, then, that the "synesthetic personality" may be the personality of someone who is more likely to "stumble"into synesthesia – although of course some of this personality (which has always been measured in adults) could also be the result of living with synesthesia for many years.

Are children deliberately choosing to use synesthesia as a learning strategy, or consciously aware of doing so? It might be that such associations are spontaneously made at a relatively sub-personal or unconscious level, and that they are strengthened and maintained because they prove useful for certain tasks (cf. Galton, 1881; Calkins, 1893). But in some cases the original impetus for making these associations could be conscious deliberation about the difficult learning problem facing the child, which she will be consciously aware of in almost all cases, since, as we saw earlier, almost all synesthetic inducers are learned in a formal setting. However awareness of a learning *problem* that one has successfully solved does not always entail awareness of the *solution* one actually used, and so we remain agnostic about the degree of deliberation or consciousness involved in the strategic development of synesthetic associations.

#### **WHAT LINKS MULTIPLE FORMS OF SYNESTHESIA?**

Different forms of synesthesia appear to be interrelated. That is, if you have one form of synesthesia you are far more likely to have other forms (Sagiv et al., 2006; Brang and Ramachandran, 2011), and there are several "clusters" of synesthetic types that are more strongly related than others (Novich et al., 2011). This interrelatedness is generally explained as stemming from a genetic predisposition to develop synesthesia that manifests itself in different forms; but we suggest another possibility, namely, that if one has successfully employed a certain learning strategy, one (or one's brain) might be more likely to use similar strategies when faced with similar problems in the future.

#### **WHY ISN'T EVERYONE A SYNESTHETE?**

Virtually everyone in modern industrialized societies has learned to use letters and calendars, but most do not have colors for each letter, or convoluted spatial forms for calendars. If synesthesia develops as part of a strategy for learning about such things, then why do we not see more synesthetes? There are a number of plausible answers to this question.

First, the development of synesthesia may be possible only during critical periods of development, when the systems responsible for processing and representing inducers and concurrents are plastic enough to allow such unusual connections to form. Our review of the adult training literature found some evidence that non-synesthetic adults can have experiences that resemble certain aspects of synesthesia after training for a long enough period of time. However the highly structured and long-lasting inducer– concurrent relationships of full-blown synesthesia may require early plasticity.

Second, a specific neurological profile that goes beyond mere plasticity may be required for the development of synesthesia. Numerous researchers have suggested, for instance, that synesthesia is caused by unusual connectivity between brain areas responsible for processing stimuli from the inducer domain and areas responsible for concurrent experiences. Such connectivity could take the form of structural differences, such as more axonal projections or more heavily myelinated projections between these areas (cf. Ramachandran and Hubbard, 2001b), or functional differences such as less inhibitory activation from other areas (cf. Grossenbacher and Lovelace, 2001), or some combination of these factors (Brang et al., 2010). A large number of studies have now confirmed that adult synesthetic brains differ in various ways from those of non-synesthetes (for recent reviews, see Hubbard et al., 2011; Rouw et al., 2011), including unusual connectivity in brain areas associated with inducer and concurrent representation, although there is still debate about whether these differences cause or are caused by the constant conjunction of inducers and concurrents (cf. Cohen Kadosh and Walsh, 2008).

Third, a specific genetic profile might be a prerequisite for the development of synesthesia. For instance, it is commonly suggested that the unusual connectivity associated with synesthesia stems from genetic mutation (e.g., Maurer, 1993; Ramachandran and Hubbard, 2001b), possibly to a gene (or genes) involved in the modulation of neural pruning during development (e.g., Baron-Cohen et al., 1993; Bailey and Johnson, 1997; Hubbard and Ramachandran, 2005). Until quite recently, a relatively simple genetic trigger for synesthesia seemed quite plausible. Synesthesia was thought to be rare, with rates as low as one in 2000 in the general population, yet almost 50% of first-degree relatives of synesthetes were reported to be synesthetes, representing a thousand-fold increase (Baron-Cohen et al., 1996; Barnett et al., 2008). Evidence suggesting a strong link between synesthesia and gender bolstered this genetic interpretation. The ratio of female to male synesthetes was reported to be as high as 6:1, and one well-cited study found an 8:1 ratio of female to male family members (synesthetic or not) of synesthetes (Baron-Cohen et al., 1996; Ward and Simner, 2005; Barnett et al., 2008). Moreover, in almost all reported cases of familial synesthesia, the trait was passed along the maternal line (Baron-Cohen et al., 1996; Ward and Simner, 2005; Barnett et al., 2008). Such findings were consistent with a simple (i.e., single-gene) X-linked pattern of inheritance, possibly one that was lethal to males *in utero* (Bailey and Johnson, 1997). However later research increased sample sizes and avoided several methodological flaws, and overturned most of these findings. While there is clearly a strong tendency for synesthesia to run in families, as has been known for over a century (Galton, 1883), synesthesia is far more common in the general population than was thought, with rates of grapheme–color synesthesia alone being placed at about 1% (Simner et al., 2006). Furthermore, there is no difference in the number of males and females in the families of synesthetes, ending speculation about X-linked lethality (Ward and Simner, 2005; Barnett et al., 2008). There is also likely little or no difference in the actual rates of female and male synesthesia (Simner et al., 2006, 2009a), previously reported differences likely stemming from differences in response biases between the sexes. Finally two direct genetic studies of synesthetes found multiple loci of interest for synesthetic inheritance that differed between the two

studies, and were consistent with multiple modes of inheritance being involved in synesthesia (Asher et al., 2009; Tomson et al., 2011).

So there is almost certainly no simple genetic story to tell, no single gene or group of genes that "turns on" synesthesia. Still, there is clearly a genetic influence on synesthetic development, or more accurately a range of genetic influences. These could take the form of influences on neural pruning or inhibition, as is favored by many researchers, but they could equally be influences of another kind, such as a multi-factor genetic influence on the"synesthetic personality"described above. Whatever the nature of the genetic influences, they contribute to the relative rarity of synesthesia.

All neurological and genetic accounts, however, have the same shortcoming: they do not explain why almost all synesthetic inducers are explicitly taught, culturally dependent, categories (Day, 2005; Rich et al., 2005; Simner et al., 2006). If synesthesia is simply the result of a hyperconnected brain, then why do almost all the connections begin with objects of formal instruction? If grapheme–color synesthesia develops from an innate link between shapes and colors (Maurer et al., 2012), why do adult grapheme–color synesthetes not report colors for all shapes? At the very least, genetic and neurological accounts need to be able to answer these questions, and we see no way of doing so without a theory that places learning at the forefront of synesthetic development.

#### **CONCLUSION**

We hope that this article provides a comprehensive resource for those researchers interested in the two-way influences between synesthesia and learning. With the wide range of evidence we have summarized for the influence of learning on synesthesia, we hope to have shown that far from being unlearned,"learning is the defining characteristic of synesthesia" (Witthoft and Winawer, 2013). The growing body of work on synesthesia's utility for learning demonstrates, in addition, that it can be exploited in both implicit and explicit learning of many different kinds.

We do not expect to have convinced all our readers of the validity of our learning hypothesis of synesthetic development, but we do hope to have sparked some interest in it. We also hope that by reviewing the two-way influences between synesthesia and learning, we have contributed to the growing recognition of the importance of learning in synesthesia.

#### **REFERENCES**


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 22 October 2013; accepted: 09 February 2014; published online: 28 February 2014.*

*Citation: Watson MR, Akins KA, Spiker C, Crawford L and Enns JT (2014) Synesthesia and learning: a critical review and novel theory. Front. Hum. Neurosci. 8:98. doi: 10.3389/fnhum.2014.00098*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2014 Watson, Akins, Spiker, Crawford and Enns. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Acquiring synaesthesia: insights from training studies

#### *Nicolas Rothen1 \* and Beat Meier <sup>2</sup>*

*<sup>1</sup> Department of Psychology and Sackler Centre for Consciousness Science, University of Sussex, Brighton, UK <sup>2</sup> Department of Psychology and Center for Cognition, Learning and Memory, University of Bern, Bern, Switzerland*

#### *Edited by:*

*Hauke R. Heekeren, Freie Universität Berlin, Germany*

#### *Reviewed by:*

*Olympia Colizoli, University of Amsterdam, Netherlands Flor Kusnir, University of Glasgow, UK*

#### *\*Correspondence:*

*Nicolas Rothen, Sackler Centre for Consciousness Science, School of Psychology, University of Sussex, Falmer, Brighton, BN1 9QH, UK e-mail: nicolas.rothen@gmail.com*

Synaesthesia denotes a condition of remarkable individual differences in experience characterized by specific additional experiences in response to normal sensory input. Synaesthesia seems to (i) run in families which suggests a genetic component, (ii) is associated with marked structural and functional neural differences, and (iii) is usually reported to exist from early childhood. Hence, synaesthesia is generally regarded as a congenital phenomenon. However, most synaesthetic experiences are triggered by cultural artifacts (e.g., letters, musical sounds). Evidence exists to suggest that synaesthetic experiences are triggered by the conceptual representation of their inducer stimuli. Cases were identified for which the specific synaesthetic associations are related to prior experiences and large scale studies show that grapheme-color associations in synaesthesia are not completely random. Hence, a learning component is inherently involved in the development of specific synaesthetic associations. Researchers have hypothesized that associative learning is the critical mechanism. Recently, it has become of scientific and public interest if synaesthetic experiences may be acquired by means of associative training procedures and whether the gains of these trainings are associated with similar cognitive benefits as genuine synaesthetic experiences. In order to shed light on these issues and inform synaesthesia researchers and the general interested public alike, we provide a comprehensive literature review on developmental aspects of synaesthesia and specific training procedures in non-synaesthetes. Under the light of a clear working definition of synaesthesia, we come to the conclusion that synaesthesia can potentially be learned by the appropriate training.

**Keywords: synaesthesia, definition, development, training, learning, acquiring, control**

## **INTRODUCTION**

Synaesthesia denotes a condition of remarkable individual differences in experience characterized by specific additional experiences in response to normal sensory input. For instance, the letter A printed in black (i.e., an inducer) may elicit a red color experience (i.e., a concurrent). Already more than 100 years ago, Bleuler and Lehmann suggested that "the disposition to secondary sensations [synaesthetic experiences] is highly hereditary" (Bleuler and Lehmann, 1881, p. 49 translated from German). However, the associative nature of the phenomenon has also led researchers to ask the question whether synaesthesia or the specific associations, respectively, may be acquired through associative learning (Kelly, 1934; Howells, 1944). Despite this longstanding interest into the developmental aspects of synaesthetic experiences the exact causation is still subject to debate and it is still not clear if synaesthesia can be acquired through training inducer-concurrent associations. In this review, we first consider developmental aspects of synaesthesia and link these to the possibility that synaesthesia can be trained. Next, we review in depth the available literature explicitly concerned with the trainability of synaesthetic experiences in order to shed light on these issues.

#### **WHEN TO SPEAK OF TRAINED SYNAESTHESIA?**

The question whether or not synaesthesia may be acquired through training requires a clear definition of synaesthesia which can be used to assess whether any potential training procedure was successful in inducing synaesthetic experiences. Currently, there is general agreement on the following constitutive characteristics of synaesthesia. Synaesthetic experiences are *involuntarily* and *automatically* triggered by a stimulus (i.e., the inducer). Although *idiosyncratic*, synaesthetic experiences are *consistent* over time within the same individual. That is, while A may elicit a red color experience for one synaesthete, it may elicit a blue color experience for another synaesthete, but it will always elicit the same color experience for a specific individual (Grossenbacher and Lovelace, 2001; Ward and Mattingley, 2006; Ward, 2013; but see Meier et al., 2014). For most synaesthetic individuals, synaesthetic experiences have perceptual qualities that go beyond mere associations (Grossenbacher and Lovelace, 2001). For instance, grapheme-color synaesthesia entails the subjective phenomenological experience of seeing internally or externally represented colors (i.e., color photisms). Synaesthetic experiences are *unidirectional* on an explicit representational level but *bidirectional* on an implicit level. That is, in grapheme-color synaesthesia, a grapheme may elicit a color experience, but the color does not elicit the experience of the respective grapheme (Brugger et al., 2004; Cohen Kadosh and Henik, 2006; Meier and Rothen, 2007; Rothen et al., 2010; but see Cohen Kadosh et al., 2007 for the report of an exceptional case with explicit bidirectional synaesthesia). Hence, to confirm the hypothesis, that synaesthesia can be induced via training, would require the trained inducers to (i) *consistently* and (ii) *automatically* elicit (iii) the associated concurrent *experience* with perceptual qualities on a subjective phenomenological basis (iv) for the great majority of the inducers' occurrences (v) over an extended time period. Herewith, we suggest a rather conservative approach to prevent the potentially premature conclusion that synaesthesia can be acquired by means of training. Specifically, according to these criteria, know-associator synaesthetes, who do not report subjective phenomenological color experiences (as opposed to see-associator synaesthetes and projector synaesthetes), but simply know their synaesthetic associations, would not be considered as genuine synaesthetes (cf., Ward et al., 2007).

#### **DEVELOPMENTAL ASPECTS**

As noted above, a genetic predisposition seems to be constitutive of synaesthesia (Baron-Cohen et al., 1996; Ward and Simner, 2005; Tomson et al., 2011). That is, synaesthesia tends to run in families (Asher et al., 2009), although individual members of a family may experience different forms of synaesthesia (Barnett et al., 2008). For instance, while one family member may experience colors for letters and numbers (i.e., grapheme-color synaesthesia), another family member may experience spatial arrangements in response to sequence based concepts such as the days of the week (i.e., sequence-space synaesthesia). However, even when synaesthesia occurs within a family, typically not all members are concerned. The presence of one form of synaesthesia in an individual tends to increase the likelihood that the same individual also experiences another form of synaesthesia (Sagiv et al., 2006b). On a neural basis, synaesthesia is associated with functional and structural changes. Grapheme-color synaesthesia, which is currently the best studied form of synaesthesia, is associated with increased structural connectivity in occipito-temporal and parietal regions (Rouw and Scholte, 2007; Rouw et al., 2011; Banissy et al., 2012; Specht, 2012). The structural changes seem to be associated with functional changes during the perception of synaesthetic color experiences (Hubbard et al., 2005; Weiss et al., 2005; but see, Hupé et al., 2012). Increased activation in hV4 (i.e., involved in human color perception) is hypothesized to reflect the perception of synaesthetic color experiences (Ramachandran and Hubbard, 2001a,b). Increased activation in parietal regions around the sulcus intraparietalis and gyrus angularis are thought to reflect binding processes between the synaesthetic inducer and the synaesthetic concurrent experience (Esterman et al., 2006; Muggleton et al., 2007; Rothen et al., 2010). Moreover, some researchers suggested that synaesthetic experiences have real perceptual qualities which may lead to a performance advantage in perceptual tasks (Ramachandran and Hubbard, 2001a; Hubbard et al., 2005; Sagiv et al., 2006a; Ward et al., 2010; but see, Mattingley et al., 2001; Edquist et al., 2006; Rothen and Meier, 2009). Others have argued that synaesthetic experiences are represented as nodes in a semantic network and that synaesthesia is not a pure perceptual condition (Meier, in press). Synaesthesia is generally regarded as a congenital condition and, as such, sometimes also termed "developmental synaesthesia" to distinguish from "metaphorical synaesthesia" (e.g., screaming colors).

However, a genetic predisposition does not necessarily mean that the respective condition will indeed develop. Even in the case where a genetic predisposition for synaesthesia eventually results in synaesthetic experiences, it is rather unlikely that the specific synaesthetic associations are determined by genes. This is especially the case because most synaesthetic experiences are triggered by cultural artifacts, as for instance in grapheme-color synaesthesia (Rich et al., 2005; Simner et al., 2009), sequence-space synaesthesia (Eagleman, 2009), and lexical-gustatory synaesthesia (i.e., words elicit taste experiences; Ward and Simner, 2003). Crucially, synaesthetic consistency is only achieved after conceptual knowledge about the synaesthetic inducer has been acquired (Simner et al., 2009). In line with this notion, it has been suggested that it is the conceptual representation of the synaesthetic inducer which triggers the specific concurrent experience (Nikolic´ et al., 2011; Rothen et al., 2013), or at least that the conceptual representation of a synaesthetic inducer is sufficient to elicit the concurrent experience (Dixon et al., 2000, 2006). Moreover, there are cases for which the specific synaesthetic associations are related to prior experiences (for example colored alphabet toys; Witthoft and Winawer, 2006, 2013). Similarly, large scale studies show that grapheme-color associations in synaesthesia are not completely random (Rich et al., 2005; Simner et al., 2005). However, these studies failed to find correlations with colors for letters and numbers in children's books (Rich et al., 2005). Crucially, also the structural neural basis of synaesthesia may be a consequence rather than a precondition for synaesthetic experiences. In line with this notion, synaesthetic experiences can be induced post-hypnotically in non-synaesthetic individuals who do not possess the structural neural basis of synaesthesia. Hence, synaesthetic experiences may be a result of functional disinhibition between relevant brain areas, but structural changes are not necessary for conscious synaesthetic experiences (Cohen Kadosh et al., 2009). Similarly, developmental synaesthetes can acquire synaesthetic experiences for novel inducers within minutes (Mroczko et al., 2009). Furthermore, the early onset of synaesthesia does not categorically exclude the possibility that synaesthetic experiences can be acquired later in life. For instance, there is evidence for a higher prevalence of synaesthesia among artists and meditators which suggests that synaesthesia may be cultivated (e.g., Walsh, 2005; Rothen and Meier, 2010b).

To summarize, a learning component is inherently involved in the development of specific synaesthetic associations, in particular when the inducers are cultural artifacts (as in grapheme-color synaesthesia). Some researchers have hypothesized that associative learning is the critical learning mechanism, where the recall event consists of implicit imagery of the synaesthetic concurrent (Albright, 2012). Hence, it seems plausible that synaesthetic experiences may be acquired, and there is already clear evidence that at least some aspects of synaesthesia can be acquired through training. Training studies may provide further insights into the mechanisms which are at play during the development of synaesthesia.

#### **TWO RESEARCH TRENDS OF PUBLIC INTEREST**

Over the last 10–20 years there was an increase in scientific publications about synaesthesia (**Figure 1**). Earlier publications were

mainly concerned with documenting specific types of synaesthesia and demonstrating their genuineness (Baron-Cohen et al., 1987). Besides this, later publications were also concerned with documenting the characteristics associated with synaesthesia. There is empirical evidence suggesting that synaesthesia is associated with a specific profile of enhanced memory performance (Yaro and Ward, 2007; Rothen and Meier, 2010a; Radvansky et al., 2011; Rothen et al., 2012; Meier and Rothen, 2013b), increased creativity (Rich et al., 2005; Ward et al., 2008; Rothen and Meier, 2010b), and increased self-rated imagery (e.g., Barnett and Newell, 2008; Meier and Rothen, 2013a). The increase in scientific publications led to documentary films (e.g., BBC) and articles in popular science magazines (e.g., Ramachandran and Hubbard, 2003; Lehrer, 2007; Bouska, 2013). Consequently, public awareness and interest in this—to the public—seemingly peculiar condition was rising. Possibly due to the positive effects associated with synaesthesia, it became a question of more general interest to what extent synaesthesia can be learned or trained. This is reflected on websites and blogs on the internet which discuss the issue and relevant findings from research studies. Moreover, a "market" exists to promote synaesthesia related products. For example, synaesthetic training courses are offered to improve creativity and memory (Samarajiwa, 2014; Söffing, 2014; cf. also, Preiser, 2011). Synaesthesia training has also been suggested as a form of psychotherapy (Synästhetische Gedanken [Synaesthetic Thoughts], 2014; cf. also, Lewis, 2012). Interestingly, even synaesthetes seem to ask how they can enhance their experiences, for example in order to increase their creative artistic output (Kann ich meine Synesthäsie weitertrainieren, damit ich sie nutzen kann? [Is it possible to train my synaesthesia for further benefits?], 2014). However, there is no scientific basis for the claims that synaesthesia training can actually provide for the cognitive profile associated with synaesthesia and its advantages (and potential disadvantages). Therefore, the question, whether synaesthesia can actually be acquired via training, is important for synaesthesia researchers and to inform the public.

Independent from synaesthesia research, cognitive training has also gained a lot of interest over the last two decades (**Figure 2**).

The correspondent scientific publications even outnumber those about synaesthesia. Cognitive training studies are mainly concerned with transfer effects. That is, whether training one cognitive process affects other cognitive processes that were not trained. For instance, recent findings suggest that working memory training can also enhance performance in intelligence tests (e.g., Jaeggi et al., 2008, 2011). Accordingly, there is great public interest in cognitive training, resulting in similar effects as the interest in synaesthesia. That is, there are numerous blogs and internet pages on cognitive training, as well as articles in popular science magazines (e.g., Sinha, 2005; Robertson, 2010). Moreover, cognitive training programmes are sold in form of computer games (e.g., Nintendo's "Dr. Kawashima's Brain Training: How Old is Your Brain?").

From this point of view, synaesthesia training studies are important to understand the plasticity of the cognitive system more generally. As there is evidence that synaesthesia is associated with cognitive benefits, and cognitive training can lead to transfer effects, it is a promising avenue for future research to investigate transfer effects of synaesthesia training.

## **ACQUIRING SYNAESTHESIA: STATE OF RESEARCH TRAINING STUDIES**

So far, seven synaesthesia training studies have been published. All were explicitly concerned with the question whether *consistent* and *automatic* concurrent*experiences* with perceptual qualities on a subjective phenomenological basis can be induced. These studies will be reviewed in depth. They are thematically and roughly chronologically ordered (see **Table 1** for a summary).

#### *Early attempts: sound-color synaesthesia via conditioning*

In a first study, it was tested whether sound-color synaesthesia can be induced artificially by means of associative learning/conditioning (Kelly, 1934). The following introspective criterion was applied to test the success of the study: "If the tones became capable of arousing spontaneous sensations or images of color in the subjects, then and only then, could it be said that a true conditioned response had been established." (Kelly, 1934). Eight different accordion tones of a complete octave and


**127**

seven different projected color squares were used to create tonecolor pairs (C and C1 were both white). Eighteen participants, all non-synaesthetes but one sequence-space synaesthete, attended a passive training procedure including multiple sessions during which the pairs were presented (tone and color simultaneously) for a duration of ∼9 s. The interval between two pairs was 1.5 s, and the scale was always presented in ascending and then descending order several times per experimental session. Over the duration of the experiment, six participants were presented with at least 2000 repetitions per stimulus pair. For the remaining participants, the minimum was 280 repetitions per pair. Once a week, participants were presented with 40 tones alone (each tone five times) and were asked to indicate the color associated with the tone, and whether they had a sensation of color for the tone. None of the participants reported any color sensations. The results showed no evidence at all that synaesthesia may be induced artificially. Interestingly, Kelly (1934) pointed out that "It has been argued by some that a purely physiological theory is sufficient without including the assumption that conditioning occurs. A crucial test of whether conditioning does supplement a physiological factor in the production of chromaesthesia [synaesthesia] could be made by attempting to destroy and change the color-tone associations in bona fide cases of colored hearing [sound-color synaesthesia]. If it could be shown that it is possible to experimentally destroy the linkages reported, it would offer almost incontrovertible evidence that conditioning had played a role in the production of synaesthesia."

Ten years later, another study was conducted with the aim to establish sound-color synaesthesia in a group of eight participants (Howells, 1944). Using a conditioning paradigm, two tones (middle C and G above) and two colors (red and green) served as stimuli. Participants were required to keep their eyes closed until the onset of the tone, to open their eyes, and to report the color which had its onset immediately after the tone. The tone was presented for the duration of 2 s, and its offset was simultaneous with that of the color. Next, participants were required to close their eyes again for the next trial. Middle C was paired with red and G was paired with green for 95% of the trials. The pairings were reversed for the remaining 5% of the trials. These stimulus pairs were presented in different predetermined random orders. Participants were informed about the correctness of the response after every trial. After 5000 trials of presenting the colors at maximum saturation, the hue was reduced for 50% of the trials (including 50% of the reversed tone-color pairings). As dependent variable, error rates were assessed for reversed tone-color pairings in sets of 50 trials. After the first 5000 trials for which almost no errors occurred, the error rates increased almost linearly to ∼17 over the course of the study. Each participant was presented with ∼30,000 trials in total. The explanation, offered by the participants for the observed effect, was that "The habitual set, or expectancy of seeing a given hue after hearing the paired tone, became so strong that it overpowered the conflicting perceptual influence of the hue actually supplied, with the result that pale green was actually seen as pale red, or vice versa" (Howells, 1944, p. 96). In order to determine whether the participants indeed experienced colors two of the participants were also tested with additional behavioral tasks (100 trials per participant and task). During one task, the participants were presented with the same stimuli as during the conditioning paradigm. However, instead of pale colors the participants were presented with a white stimulus in 50% of the cases, the other 50% of the cases consisted of saturated colors. One participant suspected that the "pale" stimuli were actually white, but reported that they still appeared in color to him. The other participant did not realize that the stimuli were white. During the other task, the participants were again presented with the original tones and the original colors. Tones and colors were presented simultaneously, one pair at a time, in random order. The two colors were presented by means of two light beams which could also be mixed via the adjustment of a slide. The task was to adjust the presented color to appear white. Suggesting an associated color experience (i.e., perceptual effect), it was found that for the lower (red) tone an over-adjustment toward green occurred, and for the higher (green) tone an overadjustment toward red occurred. Howells (1944) concluded that cumulative conditioning had led to synaesthetic experiences.

#### *Grapheme-color synaesthesia via associative learning*

It was not until more than half a century later before the next training study was conducted. The study was realized to investigate whether synaesthetic Stroop effects are a valid diagnostic criterion for synaesthesia, and whether psychophysiological consequences of grapheme-color synaesthesia can be established by training specific letter-color associations (Meier and Rothen, 2009). Twenty non-synaesthetes were trained to learn four different letter-color associations; A—red, B—green, C—yellow, and D—blue. Participants had to press one of two designated keys as quickly and accurately as possible if, at the center of the computer screen, a letter was presented in its correct color, and the other of two designated keys if a letter was presented in its incorrect color. Each session consisted of 480 trials. Each letter was presented 60 times in its correct color and 20 times in each of the other three incorrect colors in random order. Feedback about mean reaction time and proportion correct was given after every session to enhance motivation. There was one training session per day on 7 consecutive days. Every participant was presented with a total of 3360 trials, half of which were correct letter-color pairings. Mean accuracy over all training sessions was 98%.

After the training, participants were tested with a synaesthetic Stroop task and a synaesthetic conditioning task (Meier and Rothen, 2007). During the Stroop task, participants were presented with two letters. For half of the trials, the letters were presented in their congruent colors (i.e., according the trained association). For the other half of the trials, the letters were presented in an incongruent color (i.e., color associated with the other letter). A significant Stroop effect was found for the 20 trained participants. This was not the case for 20 untrained nonsynaesthetic controls. During the synaesthetic conditioning task participants were required to attend to colored squares. One of the squares was white and contained a letter associated with the color of another square by means of the training. This specific color was followed several times by a loud sound to provoke a conditioned startle reaction to that color (i.e., CS color). Neither the letter (CS letter) nor the other colors (neutral) were followed by the startling sound (**Figure 3**). Conditioning was measured by means of event-related skin conductance responses (SCRs), with higher SCRs indicating higher autonomic arousal. For CS color trials (but not for CS letter trials and neutral filler trials), SCRs were significantly increased during conditioning in comparison to a preceding habituation phase without the startling sound (**Figure 4**). That is, in contrast to genuine synaesthetes (Meier and Rothen, 2007; Rothen et al., 2010), non-synaesthetes, trained on letter-color associations, did not show a significant increase for the CS letter during conditioning. Moreover, none of the participants reported having color experiences for the trained letters. To summarize, the training was successful in creating automatic letter-color associations, as measured by the Stroop task, but not in creating synaesthetic experiences, as indicated by the absence of a synaesthetic conditioning effect and the subjective

**FIGURE 3 | Synaesthetic conditioning.** In this particular example, the letter D was associated with blue during the training. That is, blue squares not followed by the startling sound acted as CS color and the letters as CS letter. The remaining colors were neutral. Each square represents one trial with the first trial in the upper left corner and the last trial in the lower right corner (moving from left to right). Adopted from Meier and Rothen (2009).

**letter and neutral color stimuli for trained non-synaesthetes.** Error bars represent standard errors. Adopted from Meier and Rothen (2009).

phenomenological reports. Thus, this cannot be regarded as synaesthesia and hence, synaesthetic Stroop effects are not a valid diagnostic criterion for synaesthesia.

#### *Grapheme-color synaesthesia via adaptive/non-adaptive training*

In a follow-up study, the same training (i.e., non-adaptive) was compared with an adaptive training procedure (Rothen et al., 2011) to test for associative learning of automatic number-color associations, as they exist in grapheme-color synaesthesia. Forty non-synaesthetic participants were trained with either the nonadaptive or the adaptive version of the training on 10 consecutive days (*N* = 20 per training procedure). They were instructed to learn the following associations: 3—red, 4—green, 5—yellow, and 6—blue. Apart from these different grapheme-color associations, the non-adaptive training paradigm was identical to the previously introduced training (Meier and Rothen, 2009). That is, in the non-adaptive training, participants were presented with a total of 2400 match and 2400 non-match trials over the duration of the training. In the adaptive training, participants had to indicate as quickly and accurately as possible, by pressing one of four distinct keys, which color was associated with a black digit, presented centrally at the computer monitor. The color-key mapping changed on a trial-to-trial basis. Participants were provided with feedback after every trial and presented with a square in the correct color if their response was wrong. Next, they were presented with the digit in its correct hue but wrong brightness and had to indicate if the presented color was lighter or darker than the correct color. The brightness manipulation followed a staircase procedure depending on the previous response of a given color. Participants received also feedback for this response, and the digit was presented in its correct color if their response was wrong (**Figure 5**). Each of the daily sessions consisted of 248 trials. Hence, participants were presented with a total of 2480 trials over the duration of the training. For both versions of the training, mean accuracy was 96%.

Before and after the training, participants were tested with two synaesthetic priming tasks (cf., Gebuis et al., 2009). In one version of the task, participants were required to indicate a target color by pressing one of four specified keys. The target color was preceded by digits to which the associated color was either congruent or incongruent to the target. In the other version of the task, participants were required to indicate a target digit by pressing one of four specified keys. In this version, the colors were used as primes. After the training, the results revealed priming effects for both versions of the task in the adaptive training group. In the non-adaptive training group, priming was restricted to the digitdecision task. On average, the adaptive training group showed larger priming effects. However, none of the participants reported synaesthetic experiences after the training. Hence, adaptive training seems to be generally more effective in creating automatic digit-color associations than non-adaptive training, but it seems not sufficient enough in order to create synaesthetic experiences when applied for a short duration of 10 consecutive days.

#### *Swimming-style color synaesthesia via associative learning*

Using the non-adaptive training with pictograms of swimming-styles on 20 consecutive days (i.e., 4800 matching and 4800 non-matching trials), Rothen et al. (2013) tested whether swimming-style color synaesthesia can be acquired. According to the associations of a genuine swimming-style color synaesthete, a non-synaesthete learned the following associations: butterfly—red, breaststroke—blue, backstroke—white, and crawl—pale yellow. As it was the case for the synaesthete, the trained control showed priming effects in a color decision task when primed with pictograms of swimming styles and also in a swimming-style decision task when primed with colors (cf., Rothen et al., 2011). However, only the synaesthete showed a generalized conditioned response for pictograms of a swimming-style associated with the color that was coupled with a loud startling sound (i.e., synaesthetic conditioning; Meier and Rothen, 2007, 2009; Rothen et al., 2010). Crucially, the trained control did not report phenomenological color *experiences* in association with swimming-styles.

## *Grapheme-color synaesthesia via reading books with colored letters*

In a related study, Colizoli et al. (2012) asked 15 non-synaesthetic controls to read books with colored letters. That is, each participant learnt a distinct set of associations between the letters a, e, s, t and the colors red, orange, green, blue. Moreover, participants were asked to use a web applet that colored the letters on internet pages. They were instructed to use it whenever they were reading for a significant amount of time. On average, participants read 105,660 words over the course of 2–4 weeks. A Stroop task was conducted before and after the reading-training as in the study of Meier and Rothen (2009). After the readingtraining, but not before, participants showed a significant Stroop effect. The effect was stronger in its magnitude for lower case letters, which appeared more often during the course of the training than upper case letters. Moreover, a near significant correlation was found between the rating of the statement "I am experiencing color when thinking about certain letters" and the magnitude of the Stroop effect. However, this was not the case for the statement "I am experiencing color when I see certain letters." The authors also tested whether the learned associations would enhance performance in a perceptual crowding task. The task was adapted from a study in which a performance advantage was found for genuine grapheme-color synaesthetes over non-synaesthetic controls (Hubbard et al., 2005).

During a trial of this task, a unique black target letter surrounded by four identical black flanking letters was presented for a brief duration, randomly, either on the left or right side of a fixation-cross at the center of a computer screen. Participants were required to identify the target letter. There were two conditions, a set of trials for which the trained letters served as target letters, with one of the other trained letters as flankers, and a baseline condition with the letters d, f, g, and o. To test for inherent differences between the letters of the two conditions, a nontrained group of controls (*N* = 30) was tested with the same task. However, no performance advantage was found for the trained letter condition in the training group in comparison to the baseline condition and the untrained control group. Participants were asked in a surprise retest 6 months after the training to report the colors when given the letters. On average they were 98% correct at identifying the colors overall and 40% correct at actually remembering the specific associations. Similarly to previous training studies, the findings suggest that the training was successful in inducing automatic letter-color associations, but not in inducing the perceptual aspects of synaesthesia (i.e., synaesthetic *experience*).

#### *Grapheme-color synaesthesia via incidental associative learning*

In another study, Kusnir and Thut (2012) tested to what extent synaesthesia-like letter-color associations may be implicitly learned by non-synaesthetes. A visual search task with trials containing circular arrays of six colored letters around a fixation cross at the center of the computer monitor was used as a training paradigm (**Figure 6**). Participants were instructed to indicate by keypress whether the target letter (one of three pre-specified target letters, i.e., H, U, and S) was presented left or right of the fixation cross. The distractor letters were A, B, C, F, L, O, and

**FIGURE 6 | Visual search task.** Two exemplary trials. The two target letters of interest (i.e., H and U) appeared more often in a specific associated color. The other letters appeared in each color equally often. Adopted from Kusnir and Thut (2012).

P. Target and distractors were selected randomly on each trial. Each of the letters was presented in a different color (red, blue, cyan, yellow, green, and magenta). Learning of letter-color associations was established to occur incidentally via manipulating the frequency at which letters occurred in a specific color. That is, two of the target letters (i.e., H and U) were biased to appear 5 out of 6 times in a respective target color and 1 out of 6 times in any other color. The likelihood of the two color-biased target letters together, to be observed in their respective biased color (i.e., congruent), was 55.6%. The likelihood of the two target letters together to be observed in the respective color of the other colorbiased target (i.e., incongruent) was 2.2%. The remaining target letter was not color-biased and appeared in every color with equal likelihood (5.6%). However, when considered on their own, any of the target letters or target colors occurred with equally high likelihood across trials. In a first experiment, 28 non-synaesthetes were tested. To manipulate the depth of color processing, 14 participants were informed about the letter-color manipulation and 14 were kept naïve. The training consisted of a total of 1620 trials and was conducted on 2 consecutive days. In a second experiment, 22 non-synaesthetes participated and all were informed about the likelihood of the letter-color pairings. Moreover, for half the participants the two target colors of interest were opponent colors, but not for the remaining half of the participants. Notably, greater interference for opponent colors, in comparison to non-opponent colors, was reported as evidence for early stages in the visual processing of genuine synaesthetic colors (Nikolic´ et al., 2007). The training consisted of a total of 2430 trials, and the task was performed over three sessions within 1 working week.

A significant difference in search performance between congruent and incongruent letter-color pairings was found in both experiments. That is, congruent letter-color pairings facilitated target detection. Incongruent letter-color pairings impaired target detection. Crucially, color information *per se* would not have enhanced target detection in the congruent, relative to the incongruent, condition but equally across these conditions. Greater interference for incongruent targets was found in the opponent color condition than the non-opponent color condition in Experiment 2. After each of the two training sessions, participants of Experiment 1 were tested with two additional Stroop paradigms similar to those used in previous studies (Meier and Rothen, 2009; Colizoli et al., 2012). One version was conducted with saturated colors, the other version was conducted with unsaturated colors to account for the possibility of weak acquired synaesthetic color experiences. Generally, on a group level no Stroop effects were found. Further explorations revealed a Stroop effect in the deep color processing group for the faint version of the task. Crucially, however, none of the trained participants reported color *experiences* for (the trained) letters at the end of the experiments. Because of this and the weak evidence for Stroop effects in the faint version of the task, the authors concluded that the learned associations were qualitatively different to synaesthesia.

#### *Related studies*

The following four studies are important to the aim of this review because non-synaesthetes, trained on grapheme-color associations, were included. However, the participants were not explicitly asked about their phenomenological experiences. That is, the trained associations were regarded as purely semantic. The studies shall be summarized each very briefly in one paragraph. An overview and additional information can be found in **Table 1**.

Cohen Kadosh et al. (2005) tested whether training digitcolor associations in non-synaesthetes (*N* = 6) can result in implicit numerical magnitude representation for colors. At the beginning of the session, participants were presented with the specific associations. In the next task, they had to indicate as quickly and accurately as possible if the background color of the computer screen matched the color associated with a centrally presented gray digit (1–9) by pressing one of two distinct keys. Incorrect responses were followed by auditory feedback. Each digit-color combination was presented equally often. That is, the ratio between matching and non-matching conditions was 1:8. The task consisted of 20 blocks, including 81 trials each. This procedure was repeated on 5 consecutive days. The training consisted of a total of 8100 trials. Moreover, the trained participants and two grapheme-color synaesthetes completed a color congruity task (**Figure 7**), during which they had to decide

(right) is *smaller* than numerical distance of veridical digits (le)

**FIGURE 7 | Implicit numerical magnitudes.** Exemplary depiction of experimental conditions: Facilitation and interference effects due to activation of implicit numerical magnitudes via color information. The red horizontal arrows represent the virtual numerical magnitude as indicated by the colors. Adopted from Cohen Kadosh et al. (2005).

which of two colored digits represented the larger magnitude. Only the two synaesthetes showed the anticipated facilitation effect. That is, they responded faster when the actual colors (of the digits) on the screen represented a larger numerical distance in comparison to a smaller numerical distance (**Figure 7**; upper right compared to upper left). No interference effects were found (**Figure 7**; lower right compared to lower left). The results suggest that only genuine synaesthesia provides for implicit bidirectional representations.

One of the first studies designed to test for the perceptual reality of synaesthetic color experiences included a control group of ten non-synaesthetic participants with trained word-color associations (Nunn et al., 2002). The stimulus material consisted of eight different word-color associations that were similar to those of the synaesthetes. The learning procedure consisted of a 2 × 4 grid with the different colors presented on a computer monitor. Upon clicking on a color, the corresponding word was presented via headphones, the computer monitor filled with the color and remained on the monitor until another color was selected. Learning was tested by the presentation of a word via headphones for which the corresponding color had to be selected. Word presentation during testing was random. No feedback was given. Learning and testing cycles were continued until the criterion of 100% accuracy on five consecutive trials was reached. In contrast to synaesthetic experiences of word-color synaesthetes, who underwent functional Magnetic Resonance Imaging (fMRI) in the same study, imagined colors in trained controls did not activate color selective brain regions (i.e., V4 and V8). The findings suggest that the training did not lead to color associations of similar perceptual quality as synaesthetic experiences.

In order to assess conceptual and perceptual aspects of synaesthetic experiences, Brang et al. (2011) conducted an electroencephalogram (EEG) study with genuine grapheme-color synaesthetes (*N* = 12) and trained controls (*N* = 24). During the training, achromatic graphemes were presented at the center of the screen, surrounded by ten distinct colored blocks (i.e., red, orange, yellow, green, blue, purple, black, brown, white, and pink) whose position was randomized on each trial. By clicking on the colors, participants had to learn ten grapheme-color pairings in a guess-and-check fashion. Participants were trained, until they were able to correctly identify the color of each of the ten graphemes two times in a row without any errors. The duration of the training was ∼15 min on average. The synaesthetes, the trained controls, and an additional non-trained control group (*N* = 12) were tested with a contextual priming paradigm. That is, participants were presented with sentences like "The Coca-Cola logo is white and X." On a given trial, "X" was either a color patch, color word, or a grapheme, which was either congruent or incongruent with the representation of "X" (i.e., red in our example). That is, the presented graphemes were associated to either congruent or incongruent colors (relative to the semantic context) for synaesthetes and trained controls. For the non-trained control groups, the graphemes were presented in actual colors. For all groups, half of the sentences of each type ended with a congruent stimulus and half with an incongruent stimulus. Event-related potentials (ERPs) were measured for the representation of "X." Representing perceptual processes, early ERPs to graphemes were significantly affected by congruency in the synaesthete group only but not in the control groups (unless they were presented with colored graphemes). Representing semantic processes, late ERPs to graphemes were significantly affected by congruency in all groups. The groups were not differentially affected by congruency in the other conditions. Thus, the findings suggest that the training procedure in this study was not sufficient to induce synaesthetic color *experiences* to graphemes beyond the level of semantic associations.

The main goal of a related EEG study (Niccolai et al., 2012) was to compare the neural processes associated with synaesthetic priming (similar to previous studies; e.g., Rothen et al., 2011) for congruent and incongruent trials between seven genuine grapheme-color synaesthetes and seven trained controls. The controls were required to learn specific grapheme-color associations (letters and digits; average *N* = 30) with a computer based training at home on 6 consecutive days. The training consisted of (1) selecting the color patch corresponding to a grapheme in a three-alternative forced choice paradigm, (2) assigning "correctly" four different color-patches to four different graphemes, (3) reporting the grapheme associated with a presented color patch. The training was not further specified. It lasted 18 min on average. The error rate declined significantly from start to end with an average rate of 4.3%. Synaesthetes and trained controls showed similar synaesthetic priming effects (i.e., faster responses to congruent than incongruent trials). In line with Brang et al. (2011), congruency was more likely to affect early ERP components in the synaesthete group and late components in the trained control group. Hence, the results suggest that controls did not *experience* synaesthetic colors after the training.

## **DISCUSSION**

To summarize, none of the studies outlined above provided direct evidence that synaesthesia was induced by means of the specific training procedure (see Deroy and Spence, 2013 for similar conclusions). Nevertheless, the findings are promising and clearly demonstrate that certain aspects of synaesthesia can be mimicked in non-synaesthetes. Especially the differences in subjective phenomenological reports between synaesthetes and trained nonsynaesthetes and the respectively associated cognitive, behavioral, and neural profile may provide further insights into cognitive science.

The most compelling evidence that synaesthesia can be acquired later in life by means of training comes from one of the earliest studies. As described by the author, the explanation offered by the participants indeed points to *consistently* and *automatically* elicited concurrent *experiences* with perceptual qualities on a subjective phenomenological basis (Howells, 1944). Also more recent studies provide hints that synaesthesia can be acquired by means of training. For example, the finding of greater interference for incongruent targets in the opponent color condition than the non-opponent color condition may reflect a perceptual effect as documented in developmental synaesthesia (Kusnir and Thut, 2012 Experiment 2). However, an alternative interpretation, namely that opponent colors may be more easily conceptualized needs to be ruled-out in future research (cf., Deroy and Spence, 2013). Another promising result concerns the correlational findings between the magnitude of Stroop effects and the participants' rating of the statement "I am experiencing color when thinking about certain letters" in the study by Colizoli et al. (2012). However, the finding needs to be interpreted with caution as Stroop interference indicates that an association occurs automatically but allows no conclusions about the perceptual nature of the association. Moreover, the statement "I am experiencing color when I see certain letters" did not correlate with Stroop interference. Future studies should also include items such as "I do not experience colors when thinking about or seeing certain letters, but I automatically associate them with colors" to prevent suggestive questioning.

Interestingly, these studies which reported that synaesthetic *experiences* may be, at least in part, acquired later in life are also those which included the largest numbers of inducer-concurrent pairings. Howells (1944) presented his participants with ∼30,000 trials of which 95% corresponded to the to-be-learnt associations. The participants in the study of Colizoli et al. (2012) read on average over 100,000 words containing consistently colored letters (i.e., no incongruent letter-color pairs). The success of these training studies follows logically from the fact that every training procedure needs to act against lifelong normal experiences. None of the other studies included more than 10,000 training trials. Moreover, it seems that explicit training procedures (Howells, 1944) lead to stronger effects over shorter time periods than implicit training procedures (Colizoli et al., 2012). Nevertheless, it seems that also implicit training procedures are fruitful to result in potential perceptual effects (Colizoli et al., 2012; Kusnir and Thut, 2012). However, future studies will need to rule out the alternative explanations in the previous paragraph before firm conclusions can be drawn.

The ratio between to-be-learned and not-to-be-learned associations seems to be another crucial factor of the training procedure (cf., **Table 1**). Generally, there is broad agreement between the different studies that even a relatively short period of training is sufficient to mimic certain aspects (but not the *experience*) of synaesthesia such as Stroop interference for congruent and incongruent inducer stimuli (e.g., Meier and Rothen, 2009; Brang et al., 2011; Rothen et al., 2011). The only exception to this is a study in which all potential combinations of inducers and concurrents were presented equally often. Consequently, the ratio was to the disadvantage of the to-be-learned associations (Cohen Kadosh et al., 2005).

Despite the promising results, even very extensive training durations do not necessarily guarantee to result in phenomenological synaesthetic *experiences*. Probably the strongest overlearned digit-color associations, reported for a non-synaesthetic control, evolved over an extensive period of 8 years through the use of cross-stitch patterns (Elias et al., 2003). In a synaesthetic consistency test the trained control was 100% consistent and the synaesthete was 98% consistent. The control and the synaesthete were also tested using fMRI. Naming colored single-digit numbers revealed a behavioral Stroop effect and similar brain activations for congruently and incongruently colored numbers in the control and the synaesthete. As the control was not interviewed about her subjective phenomenological experiences for numbers, it is open to debate to what extent these resembled those of the synaesthete. However, brain activation in the control and the synaesthete differed in two other tasks. These tasks consisted of simple arithmetic problems, which were either created from dice patterns or presented auditorily, and had to be solved silently. It is open to debate whether this was due to different strategies in solving the specific tasks or due to different phenomenological experiences related to the colors associated with the numbers presented during the tasks.

"Training" during childhood seems more promising and can result, but does by no means guarantee to result, in phenomenological synaesthetic experiences. There is evidence from grapheme-color synaesthesia that repeated exposure to colored childhood toys (e.g., letter refrigerator magnets) early in life leads to synaesthetic *experiences* (Witthoft and Winawer, 2006, 2013) as the following statement indicates: "this insistence on the perceptual nature of synesthesia in at least some cases [including two of those reported here (with synaesthesia acquired from letter refrigerator magnets)] has been invaluable in demonstrating that the color associated with a grapheme can have a great deal more specific content than just associating a letter with a color name" (Witthoft and Winawer, 2013, p. 6). In contrast, a case of monozygotic twins, who both acquired digit-color associations very early in life from a number jigsaw puzzle, does not fulfill our conservative criteria for acquired synaesthesia. When tested at the age of 12, they exhibited a behavioral Stroop effect in a color naming task with colored-digits. However, "they do not report photisms or any sense of perceiving a color, they simply know that zero is pink." (Hancock, 2006, p. 149). These cases suggest that the development of phenomenological synaesthetic experiences may be more dependent on internal contingencies. In contrast, the nature of the exact association may be more dependent on external contingencies. For instance, training may only influence the exact pairings (i.e., which color is paired to which letter) and may only be effective given certain internal circumstances (e.g., genetic predisposition).

There are other potential candidates for internal contingencies besides a genetic predisposition. As already mentioned in the introduction, synaesthesia is associated with a specific profile of enhanced memory performance (Yaro and Ward, 2007; Rothen and Meier, 2010a; Radvansky et al., 2011; Rothen et al., 2012; Meier and Rothen, 2013b), increased creativity (Rich et al., 2005; Ward et al., 2008; Rothen and Meier, 2010b), and increased self-rated imagery (e.g., Barnett and Newell, 2008; Meier and Rothen, 2013a). Each of these factors alone or in combination may contribute to the emergence of synaesthesia. For example, if in a particular situation, a person is more likely to assign colors to letters, enhanced imagery may result in vivid color experiences and good memory will eventually give rise to the subsequent retrieval of these vivid color experiences during letter processing. This is also in line with the notion that child synaesthetes become more consistent over time or lose the condition entirely (Simner et al., 2009; Simner and Bain, 2013). Moreover, it may be the reason for why no single genetic factor has been identified to cause synaesthesia. The genetic factors to determine the phenotype of synaesthesia may be those which determine the upper limits of creativity, imagery, and memory.

Hence, future training studies may want to account for factors such as creativity, imagery, and memory to further our understanding about the development of synaesthesia. Similarly, it may be worth investigating potential transfer effects of aspects which can be mimicked in training studies. For instance, consistent and automatic associations between graphemes and colors enrich the semantic network and hence, may provide for a richer world of experiences which, in turn, might be one of the features at the core of the benefits of synaesthesia (Meier, in press). Moreover, as some associations may be more easily acquired than others, future training studies should attempt to train other forms than grapheme-color synaesthesia (cf., Deroy and Spence, 2013). Even if it may not be possible at all to acquire synaesthetic experiences, the relevant studies are still informative for cognitive science. For instance, training studies, in combination with neuroimaging and psychophysiological techniques, proved useful to gain insights about phenomenological differences reported by developmental synaesthetes and trained controls. In ERP (Brang et al., 2011; Niccolai et al., 2012) and fMRI (Nunn et al., 2002) studies, synaesthetic experiences were associated with early sensory processes/brain regions in contrast to trained associations in non-synaesthetic controls. Similarly, SCR studies proved useful to contrast synaesthetic *experiences* and semantic associations (Meier and Rothen, 2009; Rothen et al., 2013). Related to this, future studies could aim at increasing the effect of the training by providing bio-feedback.

Before we conclude, we would like to point out that, by considering the very same empirical evidence, a more negativistic view could be adopted which has led others to conclude that "synaesthesia training" and similar terms are not justified (Deroy and Spence, 2013). However, we regard exactly this term as most appropriate because it reflects the ultimate goal of the studies in question to train *specific* inducers-concurrent pairings with perceptual qualities on a subjective phenomenological basis.

#### **CONCLUSION**

The message for synaesthesia researchers and the interested general public is that there is no solid evidence that synaesthesia can be acquired by training. However, this does not exclude the possibility that synaesthesia can be learned by the appropriate training. Future research is necessary to develop more efficient training procedures. Nevertheless, previous research has clearly shown that some typical aspects of synaesthesia can be learned easily. Future studies will also need to address whether synaesthesia training may lead to transfer effects, such as enhanced memory performance, more vivid imagery, or more creative ideas.

#### **AUTHOR NOTE**

Nicolas Rothen is supported by the Swiss National Science Foundation (Grant PA00P1\_145370/1).

#### **REFERENCES**


and non-synaesthetic populations. *Cogn. Neuropsychol.* 22, 1069–1085. doi: 10.1080/02643290500200122


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 29 September 2013; accepted: 12 February 2014; published online: 03 March 2014.*

*Citation: Rothen N and Meier B (2014) Acquiring synaesthesia: insights from training studies. Front. Hum. Neurosci. 8:109. doi: 10.3389/fnhum.2014.00109*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2014 Rothen and Meier. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## *Olympia Colizoli\*, Jaap M. J. Murre and Romke Rouw*

*Department of Psychology, Brain and Cognition, University of Amsterdam, Amsterdam, Netherlands \*Correspondence: o.colizoli@uva.nl*

#### *Edited and reviewed by:*

*Nicolas Rothen, University of Sussex, UK*

**Keywords: synesthesia, grapheme-color, training, development, learning, memory**

There is a current debate over the possibility and validity of synesthesia training experiments (Deroy and Spence, 2013; Rothen and Meier, 2014). In order to test whether a trainee should be considered to have acquired a trained form of synesthesia, a precise definition and specific diagnostic criteria of *synesthesia* are necessary. There is currently not one specific checklist available including all specific diagnostic methods and criteria, exacerbating the determination and interpretation of differences between (potentially) trained and genuine synesthesia. In order to facilitate communication surrounding these issues, we propose a practical guideline for diagnosing the specific characteristics that are typical of *grapheme-color synesthesia* (GCS). These guidelines can be applied to developmental cases of GCS, cases of trainees who may show synesthetic traits, and other types of acquired forms of GCS at the level of a *single individual*.

Researchers have tried to train synesthesia since at least 1934 (Kelly, 1934), and several studies have been done in recent years (for reviews see: Deroy and Spence, 2013; Rothen and Meier, 2014). Leaving trained, acquired, or induced types of synesthesia aside, it is important to acknowledge the extent of interindividual differences between synesthetes (even within one sub-type), making a generally accepted definition difficult to reach. For example, an individual may report that letters automatically induced the conscious experience of color since childhood, however, these colors may not be *consistently* mapped to each letter. Such an individual would fail the "test of genuineness," while he or she may still meet the other defining characteristics of synesthesia, such as the conscious experience of color in the absence of physical color. This is one example of why the definition of *developmental synesthesia* is a topic of active debate in the literature (Cohen Kadosh and Terhune, 2012; Eagleman, 2012; Simner, 2012). The need for a consensus on the specific defining characteristics of synesthesia becomes only more prominent when trying to compare trainees to synesthetes.

Characteristics of GCS at the individual vs. group level must be distinguished. Group-level effects related to synesthesia cannot (yet) be used as diagnostic tools. If two groups differ *relatively* from each other on a certain score or measurement, this does not tell us at an *absolute* level whether "score X on task A" or "measurement Y in trait B" of an individual implies that he or she is necessarily a synesthete. The nature of such studies does not allow one to assume that a certain measurement is *unique* to synesthesia. For example, it has been shown that GCS is associated with increased structural connectivity compared to controls in the temporal cortex at the *group-level* (Rouw and Scholte, 2007). However, the absolute value measured for white-matter connectivity in a particular individual could be influenced by different unknown causes. Although measurements can made at the individual level (see **Table 1**), experiments designed to test group-level characteristics related to GCS by definition cannot be used as diagnostic criteria, such as memory benefits (Yaro and Ward, 2007; Rothen and Meier, 2010; Rothen et al., 2012; Pritchard et al., 2013; Terhune et al., 2013), differences in visual processing (Terhune et al., 2011; Brang et al., 2012; Banissy et al., 2013), the reported vividness of visual mental imagery (Barnett and Newell, 2008), distinct cognitive styles (Meier and Rothen, 2013) or differences in neuroanatomy (for a review see: Rouw et al., 2011). Similarly, idiosyncrasy (the fact that grapheme-color mappings differ *between* individuals) cannot be used as a marker at the individual level; if two synesthetes have nearly identical mappings, they are not excluded from being considered synesthetes (Witthoft and Winawer, 2013). Additionally, the low prevalence of synesthesia is a group-level characteristic that is not diagnostic at the individual level. Most neurobiological markers related to GCS (Rouw et al., 2011) and the presence of certain genetic markers (Asher et al., 2009; Tomson et al., 2011) are related to the group-level. Individual-level traits are sometimes confused with inclusive criteria for GCS. For example, GCS *tends* to be unidirectional at a conscious level (i.e., graphemes elicit color experiences, but color experiences do not elicit conscious grapheme experiences) and bidirectional at an unconscious level (Knoch et al., 2005; Cohen Kadosh and Henik, 2006; Cohen Kadosh et al., 2007; Gebuis et al., 2009; Weiss et al., 2009; Rothen et al., 2010). Still, an instance of *conscious* bidirectional GCS does not exclude an individual from being considered a synesthete when the other defining characteristics are met (e.g., Cohen Kadosh et al., 2007).

Diagnosing synesthesia *in a single individual* is based on a variety of characteristics. Concerning diagnosing "trained" forms of synesthesia, Rothen and Meier (2014) state: "Hence, to confirm the hypothesis, that synesthesia can be induced via training, would require the trained inducers to (i) *consistently* and (ii) *automatically* elicit (iii) the associated concurrent*experience* with perceptual qualities on a subjective phenomenological basis (iv) for the great majority of the inducers' occurrences (v) over an extended time period" (italics are the authors' own). We propose to extend their definition with a diagnostic criteria checklist.


#### **Table 1 | Proposed diagnostic criteria for grapheme-color synesthesia at the level of a single individual.**

*(Continued)*

#### **Table 1 | Continued**


*(Continued)*

#### **Table 1 | Continued**


*Methods, criteria, and references (when available) are listed for testing the necessary dimensions related to diagnosing synesthesia along the major axes: graphemecolor synesthesia (seven dimensions) and developmental grapheme-color synesthesia (nine dimensions). Some of the dimensions have multiple methodologies for testing the criteria. More research is needed to test whether different methodologies used within one dimension provide corresponding results. We propose at least one criterion per dimension should be met for an individual to be considered to have grapheme-color synesthesia.*

We propose a guideline for diagnosing GCS based on dimensions along major axes for GCS (**Table 1**). Such a checklist can aid in the diagnosis of GCS (whether trained, acquired or developmental in nature). In this checklist, we present methods, criteria and references for the measurement of each dimension whenever possible. We propose that the presence of at least one result within each of the dimensions in **Table 1** to be the necessary criteria for a diagnosis of GCS at the individual level. Several dimensions have multiple methodologies for testing the criteria. An important question that cannot yet be answered is whether the different methodologies give congruent patterns of results. For example, do individuals who report having stronger perceptual experiences also show larger perceptual crowding effects? Such issues are particularly relevant when testing trainees, where differences between measurements may emerge, such as the presence of a Stroop effect in the absence of conditioned response (Rothen and Meier, 2009).

The **Consistent** nature of the mappings involves the *Percentage* of mappings that do not change over a *Duration* of time (e.g., Baron-Cohen et al., 1987; Asher et al., 2006), or the *Level* of consistency as measured in color distance between repetitions of identical trials of a test (Eagleman et al., 2007; Rothen et al., 2013a). Although the consistency criterion is debated, the percentage and duration of consistency have traditionally been considered the "test of genuineness," because of the inability of control participants to perform at the same level as synesthetes when all task demands are in their favor (e.g., Baron-Cohen et al., 1987; Asher et al., 2006). If the presence of synesthesialike behavior or experience is temporary (e.g., drug use or hypnosis), the individual will be excluded over time by the consistency criterion.

The **Automatic** nature of synesthesia refers to the involuntary nature of the experience; it cannot willfully be "turned on or off." (This involuntary nature does not necessarily imply that GCS is preattentional or unconsciously evoked, e.g., see Mattingley, 2009). Automaticity can be verified in behavior, with the commonly used "synesthetic Stroop task," (Wollen and Ruggiero, 1983) or with a validated questionnaire (Rothen et al., 2013b), and in physiology as measured with pupil diameter (Paulsen and Laeng, 2006) or a color-sound conditioning paradigm (Meier and Rothen, 2009).

The **Conscious** nature of the experience of a percept in the absence of its external stimulation is perhaps the hallmark of synesthesia. Perception can be conscious or unconscious in neurobiological terms (e.g., Lamme and Roelfsema, 2000). Verifying the conscious perception of a synesthetic experience is typically the starting point of recruiting synesthetes to participate in a study. Synesthetic experiences are verified to be *conscious* by interview with the individual. Conscious experiences have been found related to consistent experiences (Simner et al., 2006). Trainees can be assessed with specifically designed questionnaires (Colizoli et al., 2012, 2014).

The **Perceptual** nature of the synesthetic experience can be objectively verified by using a variety of psychophysical visual tasks (e.g., Palmeri et al., 2002; Hubbard et al., 2005; Kim et al., 2006; Nikolic et al., 2007 ´ ). Confirmatory findings where obtained with neuroimaging techniques (e.g., Nunn et al., 2002; Elias et al., 2003; Steven et al., 2006). Not all sub-types of synesthetes consistently show perceptual effects of synesthetic color when measured objectively (e.g., Hubbard et al., 2005; Rothen and Meier, 2009; Ward et al., 2010; Rich and Karstoft, 2013), and this is a point of discussion in diagnosing GCS (Rothen and Meier, 2014).

**Bandwidth** refers to the extent of the mappings between modalities and this variability leads to the idiosyncratic nature of GCS at the group level (Asher et al., 2006; Rothen and Meier, 2013). *Inducer-Bandwidth* refers to the proportion of synesthetic inducers compared to the number of possible inducers. *Concurrent-Bandwidth*, which has received relatively less attention in the field, refers to the proportion of synesthetic concurrents compared to the number of possible concurrents.

**Perceptual Presence**, which is the feeling that an experience is "real" in the same way that an event or object in the outside world is real, is characteristically *lacking* for synesthetes (Rouw et al., 2013; Rouw and Ridderinkhof, 2014; Seth, 2014). A synesthete typically does not get confused between sensations reflecting events in the outside world, and sensations caused by their synesthesia. Thus, for a synesthete, an inducer has perceptual presence, but the concurrent does not. This distinction is similar to the distinction drawn between synesthetic experiences and hallucinations (e.g., Cytowic, 2002; Sagiv et al., 2011); synesthetic experiences are notably simpler, more regular and more predictable than hallucinations.

Lastly, an important factor to consider that is often neglected is the "**Expectancy Effect**," which is the reactivity on the participants part when they behave (consciously or unconsciously) in the way they believe they are expected to behave based on the characteristics of the study. This is an important and possibly confounding factor for any study where synesthesia is evoked (e.g., through learning, suggestion, or hypnosis). It may be the case that participants have guessed the purpose of training paradigms and then behave accordingly. Therefore, none of the effects related to "synesthesia" in the dimensions listed in **Table 1** may be caused by the expectancy effect to be considered GCS.

Experiencing synesthesia since early childhood would be necessary for inclusion as **Developmental GCS**, but by definition would not be necessary for inclusion as trained or acquired GCS. Developmental GCS occurs in the absence of pathology, such as disease, drug use or mental disorders (Baron-Cohen and Harrison, 1997; Cytowic, 2002). Synesthetes differ from non-synesthetes in biological aspects such as functional and structural brain differences and the presence of certain genetic markers (Asher et al., 2009; Tomson et al., 2011). As previously mentioned, two main issues make the use of these markers unreliable for diagnosing synesthesia at the individual level. First, the group-level nature of such studies does not allow for inference at the individual level. Second, differences found at the neural and genetic levels may arise due to unknown causes in addition to causes that may be unique to synesthesia. Perceptual training paradigms may be suited to probe these underlying neural and genetic mechanisms.

Paradigms directed at training synesthesia can help to answer some open questions pertaining to the development of synesthesia, learning and memory, such as: Is there a critical period for the perceptual development of synesthesia? As far as we know, there are no developmental neuroimaging studies on synesthesia or training studies in children in potential critical periods of development. The brains of children show high levels of neuroplasticity, meaning high sensitivity to forming structural changes (Schlaug et al., 2009). It has been shown that specific synesthetic colors stem directly from childhood toys (Witthoft and Winawer, 2013), but this seems to be the exception, not the rule (Rich et al., 2005). Understanding the developmental pattern in synesthesia will help characterize the interaction between genes and environment in perceptual development.

In sum, we feel that trying to train "synesthesia" is a win-win situation. If GCS can be trained, we will have gained an understanding about who is more likely to become a synesthete and how. If GCS cannot be trained, we will still have gained an understanding about what the necessary and sufficient conditions must be in order to develop it. Our opinion is that trainees should not be considered to have acquired GCS *per se* unless the proposed criteria are met and should always be considered to be distinct from developmental synesthetes. We ask for and welcome the expansion and improvement of this diagnostic checklist. A commonly used model that can be both diagnostic and flexible is necessary for reaching a consensus in the field on the definition of (trained?) GCS.

## **REFERENCES**


brain-behavior correlations. *Neuron* 45, 975–985. doi: 10.1016/j.neuron.2005.02.008


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 22 April 2014; accepted: 13 May 2014; published online: 04 June 2014.*

*Citation: Colizoli O, Murre JMJ and Rouw R (2014) Defining (trained) grapheme-color synesthesia. Front. Hum. Neurosci. 8:368. doi: 10.3389/fnhum.2014.00368 This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2014 Colizoli, Murre and Rouw. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Can grapheme-color synesthesia be induced by hypnosis?

#### **Hazel P. Anderson1,2\*, Anil K. Seth1,3 , Zoltan Dienes 1,2 and Jamie Ward1,2**

<sup>1</sup> School of Psychology, University of Sussex, Brighton, UK

<sup>2</sup> Sackler Centre for Consciousness Science, University of Sussex, Brighton, UK

<sup>3</sup> School of Engineering and Informatics, University of Sussex, Brighton, UK

#### **Edited by:**

Beat Meier, University of Bern, Switzerland

#### **Reviewed by:**

Mirko Farina, Macquarie University, Australia Etzel Cardeña, Lund University, Sweden

#### **\*Correspondence:**

Hazel P. Anderson, School of Psychology, University of Sussex, Pevensey Building, Falmer, Brighton, BN1 9QH, UK e-mail: h.anderson@sussex.ac.uk

Grapheme-color synesthesia is a perceptual experience where graphemes, letters or words evoke a specific color, which are experienced either as spatially coincident with the grapheme inducer (projector sub-type) or elsewhere, perhaps without a definite spatial location (associator sub-type). Here, we address the question of whether synesthesia can be rapidly produced using a hypnotic color suggestion to examine the possibility of "hypnotic synesthesia", i.e., subjectively experienced color hallucinations similar to those experienced by projector synesthetes. We assess the efficacy of this intervention using an "embedded figures" test, in which participants are required to detect a shape (e.g., a square) composed of local graphemic elements. For grapheme-color synesthetes, better performance on the task has been linked to a higher proportion of graphemes perceived as colored. We found no performance benefits on this test when using a hypnotic suggestion, as compared to a no-suggestion control condition. The same result was found when participants were separated according to the degree to which they were susceptible to the suggestion (number of colored trials perceived). However, we found a relationship between accuracy and subjective reports of color in those participants who reported a large proportion of colored trials: trials in which the embedded figure was accurately recognized (relative to trials in which it was not) were associated with reports of more intense colors occupying a greater spatial extent. Collectively, this implies that hypnotic color was only perceived after shape detection rather than aiding in shape detection via color-based perceptual grouping. The results suggest that hypnotically induced colors are not directly comparable to synesthetic ones.

**Keywords: hypnosis, synaesthesia/synesthesia, colour/color, embedded figures test, consciousness, mental imagery**

## **INTRODUCTION**

Individuals with grapheme-color synesthesia experience a color (concurrent) when viewing particular letters, numbers or grammatical symbol (inducer). The triggered color experience is automatic (Mattingley et al., 2001) and the concurrent color is consistent over time (Simner et al., 2005). The developmental form of synesthesia emerges early in life (Simner et al., 2006), is associated with genetic differences (Tomson et al., 2011), and also structural and functional differences within the brain (Hubbard et al., 2005; Rich et al., 2006; Rouw and Scholte, 2007; Weiss and Fink, 2009). Early development may not be the only pathway for the emergence of synesthetic experiences (unless, of course, one chooses *a priori* to limit the term "synesthesia" to particular causal mechanisms). It has long been known that synesthesia can be acquired as a result of sensory loss (Armel and Ramachandran, 1999) or temporarily after taking hallucinogenic drugs (Sinke et al., 2012; Luke and Terhune, 2013). More recently, synesthesia has been reported to arise after brain damage (Ro et al., 2007) and it has also been suggested that synesthesia may arise, in blind individuals, after expertise with sensory substitution technology (Ward and Wright, 2012; see also Ward and Meijer, 2010; Farina, 2013). Finally, it has been claimed that synesthesia can be induced by hypnosis in hypnotically suggestible individuals (Cohen Kadosh et al., 2009). In the present study we reexamine this claim using an "embedded figures" test that has been widely used in grapheme-color synesthetes. The benefit of this task is that evidence for synesthesia type behavior would be measured through task improvement, rather than through deficits, which are easier to produce through task compliance. The issue is of theoretical importance because positive findings would suggest that synesthesia can (at least in some circumstances) arise from purely functional—rather than structural—brain changes (Grossenbacher and Lovelace, 2001) and, moreover, that hypnosis can create novel perceptual abilities.

Hypnosis is able to alter the phenomenological properties of participants' subjective experience (Kihlstrom, 2013; Oakley and Halligan, 2013). The process of hypnosis can be divided into induction and suggestion stages. In the induction stage, a putative "hypnotic state" is induced, or expectations for experiences are heightened, or the subject is simply alerted that the context is appropriate for a certain sort of response (e.g., Oakley and Halligan, 2009; Kirsch, 2011); in the second stage, suggestions are given to the participant to experience a (potentially) wide range of physical and perceptual experiences. There is considerable variability both in individual hypnotic susceptibility (Bowers, 1998) and in the range of experiences that can be induced. Importantly, perceptual hallucinations (e.g., hearing music) can be induced in many participants (Bowers, 1998), providing a potential link to synesthesia.

Although there is general consensus that hypnosis can alter a participant's subjective experience and this can then cause behavioral changes (Kihlstrom, 2013), the neural processes underlying the functional changes corresponding to hypnotically induced perception remain poorly understood. One class of theories postulates that highly hypnotizable people can perform tasks when hypnotized that they could not do otherwise; for example, distort perception so that they actually can see non-existent objects in a way they could not imagine (e.g., Brown and Oakley, 2004) or fail to perceive stimuli that would otherwise impinge on their awareness (e.g., pain, Hilgard, 1986). Another class of theories postulates that highly hypnotizable people cannot do anything hypnotically that they could not do anyway (e.g., Sarbin and Coe, 1972; Spanos, 1986). One way of characterizing the latter class of theory is in terms of "cold control" (Dienes, 2012), which postulates that the defining feature of acting hypnotically is simply the incorrect meta-cognition that one is not intending the (motor or cognitive) action. For example, hypnotic hallucination of an object on this account is imagining the object, but without realizing that one is deliberately creating a visual representation: It appears to occur from other causes and thus appears as perception (Dienes and Perner, 2007). That is, according to cold control, hypnotic responding involves no new abilities, just the sense that an action is happening by itself. The two classes of theory can be tested by using suggestions for abilities not already possessed by subjects: If the subject gains abilities they did not have, the second class of theory is refuted. As we will argue, suggestions for synesthetic experiences can serve this function. Synesthetic experience has some perception-like qualities that may enable enhanced performance on some tasks (Ramachandran and Hubbard, 2001) so the question arises whether hypnotically-induced synesthetic experience is more perception-like than imagined synesthetic experience.

The generation of synesthetic experience appears automatic, and automatic processes are partially defined by the difficulty in controlling them. So the question is raised whether control of synesthetic experience might be greater hypnotically rather than non-hypnotically. One developmental form of graphemecolor synesthesia has been temporarily reduced through hypnotic suggestion. Terhune et al. (2010) abolished phenomenological synesthetic experiences in a participant AR, a synesthete who experiences colors when viewing faces. She had to name the colors of face stimuli which were presented in either congruent or incongruent colors. Stroop-like interference effects were evident in comparison to controls in both reaction times (RTs) and event related potentials (ERPs) at baseline, however, after a post hypnotic suggestion these were no longer evident. This indicates the relevance of synesthesia to testing theories of hypnosis, as well as for hypnosis for testing theories of synesthesia. We will consider the converse case to that of Terhune et al., namely, of hypnotically suggesting a type of synesthesia in people who did not previously experience it.

To create a "hypnotic synesthesia" one can use the suggestion that when seeing (for example) the letter A the participant will see a special red color on the page. Supporting this idea, the phenomenological perception of color (or no color) has been manipulated using suggestion to add or drain color from patterned stimuli (Kosslyn et al., 2000). High but not low susceptible participants all reported being able to see gray-scale stimuli as colored, and colored stimuli as gray-scale when given hypnotic suggestion to do so. Further, positron emission tomography (PET) indicated changes in cerebral blood flow for left and right fusiform areas (perhaps corresponding to human V4) when hypnotic induction was used; however, non-hypnotically imagining the color changes produced significant changes only in right fusiform cortex. Hypnosis appeared to influence activity in color-sensitive areas of the brain in a way imagination alone could not.

The Kosslyn et al. (2000) study highlights the issue of demand characteristics in hypnosis research, and the tendency of subjects to either "hold back" or try harder in different conditions to produce the pattern of results they perceive as desired (Orne, 1962; Spanos, 1986). Specifically, the suggestion was more strongly worded in the hypnosis rather than the imagination condition in order that subjects would not confuse the two conditions and "slip into trance" in the imagination condition. Kirsch et al. (2008) presented the identical color adding or draining suggestions with or without hypnotic induction, and obtained substantial and near equivalent changes in color perception in both conditions. (Further, subjects rated themselves as clearly not hypnotized in the non-hypnotic condition, indicating that slipping into trance was not a problem.) Thus, when demand characteristics were more nearly equalized the difference between hypnosis and non-hypnosis in subjective experience greatly diminished. Further, McGeown et al. (2012) showed that similar activation in visual areas was produced in both the hypnotic and nonhypnotic conditions. That is, as cold control theory predicts, hypnotic hallucination with the suggestions used by Kosslyn et al. (2000) may involve the same visual abilities as imagination, with the difference being purely metacognitive (Dienes, 2012).

There has been one previous attempt to hypnotically induce grapheme-color synesthesia (Cohen Kadosh et al., 2009). Cohen Kadosh et al. assigned colors to digits either through posthypnotic suggestion or learnt association (e.g., 5 = green). Participants were required to search for an achromatic (black) grapheme against a colored background. The results showed that search was impaired when the suggested color was congruent with the background. However, this result reflects a deterioration in performance which is easier to simulate (explicitly or implicitly) than an improvement. Cohen Kadosh et al. requested control groups to associate the colors with the digits in non-hypnotic contexts; the non-hypnotic request had no effect on the search task. However, a non-hypnotic request carries different demand characteristics from a hypnotic or post-hypnotic suggestion. It is also unclear from this study whether the performance on the task does indeed resemble that found in developmental grapheme-color synesthesia. To our knowledge only one developmental grapheme-color synesthete has been tested on a version of this task (Smilek et al., 2001). While performance of this synesthete showed the same trend as the "hypnotic synesthetes" (i.e., worse performance on congruent trials, about 20%, compared to incongruent trials, about 90%), they were far from equivalent in other respects. Perhaps surprisingly, the manipulation had only a mild effect on the synesthete (88% correct for congruent and 96% for incongruent conditions) but a drastic effect on the hypnotized non-synesthetes. Although direct comparison of proportion correct trials is difficult due to differences in the task, the comparison is striking as the synesthete completed a more difficult task than the non-synesthetes, being required to provide a specific grid location for the target grapheme amongst distracters. Furthermore, although behavioral similarities between developmental and hypnotic synesthetes are informative of cognitive processes, they don't provide detail of the phenomenological experience of the participant, an aspect which requires more attention in hypnotic synesthesia research as well as more generally in neuroscience (Lifshitz et al., 2013).

In the present study, we re-examine whether hypnotically induced synesthetic colors can lead to facilitated performance on the Embedded Figures Test. Although the test itself (and the interpretation of the results) is not without controversy, it has the advantage of predicting that synesthetic (or hypnotically hallucinated) colors should facilitate rather than impair performance on a difficult task, as in the previous study. Moreover, the test has been utilized in several previous studies involving grapheme-color synesthetes providing useful benchmark comparisons. Ramachandran and Hubbard (2001) showed synesthetes arrays comprising of different graphemes (e.g., 5 s and 2 s) such that one of the graphemes could be grouped together to form a shape (e.g., a triangle made of 2 s).<sup>1</sup> The task was to identify the global shape, from four alternatives, given a limited viewing time of 1 s. Ramachandran and Hubbard (2001) found their two synesthetes to be significantly more accurate in identifying the embedded shape than controls. They later called this effect "popout" (Hubbard et al., 2006).

The effect was partially replicated by Hubbard et al. (2005) who noted that the "pop out" effect was not as great as would be expected for true color. Rothen and Meier (2009) however did not find an accuracy advantage for synesthetes in comparison to controls for the same task. Ward et al. (2010) partially supported the original findings with their larger scale replication study involving a sample of 36 synesthetes. Synesthetes' accuracy at detecting embedded shapes was significantly higher than controls, though detection rates (41% for synesthetes) remained significantly below that corresponding to true "pop out". In this study, participants were also required to rate the phenomenal vividness of the synesthetic color and to indicate what percentage of the digits appeared as colored. Importantly, the greatest performance benefits were found for those synesthetes who experienced a large proportion of the array as colored. This was interpreted by proposing that synesthetic colors may facilitate local grouping within a spatial window of attention but that synesthetic colors do not enable pre-attentive pop-out. The latter interpretation may explain why other studies, based on more standard visual search paradigms, have often failed to find any benefit of synesthetic colors in detecting a target achromatic singleton grapheme (Edquist et al., 2006; Sagiv et al., 2006). That is, synesthesia may assist local grouping of elements on the basis of color (facilitating the embedded-figures test) but synesthetic color may not enable pre-attentive pop-out (on more standard visual search).

As noted by Ward et al. (2010), grapheme-color synesthetes differ in the extent to which they perceive (or notice) their synesthetic colors during this task. Synesthetes classed as "projectors", i.e., who report their colors in the spatial location of the grapheme, were more likely to report colors (and showed a trend to do better overall on the task). The reasons for this are not completely understood (Ward et al., 2007, 2010). Nevertheless, for the present purposes we decided to optimize the chances of obtaining a significant result by instructing our highly hypnotizable participants to project colors onto the array of graphemes. If hypnotic suggestions can create grapheme-color synesthesia then hypnotically hallucinated colors will facilitate performance on this task (relative to a no-suggestion control condition). We also ask, if hypnotic grapheme-color synesthesia can be induced, are the perceptual reports similar to those of natural synesthetes in regards to the vividness and percentage of colored digits?

## **METHOD**

A counterbalanced two (condition; hypnotic suggestion vs. nosuggestion) by four (duration; 1, 2, 3 and 4 s) within subjects design was used.

## **PARTICIPANTS**

Fourteen participants aged 18–42 (*M* = 23.2, SD = 6.3) were recruited through the University of Sussex Hypnotic Susceptibility Register. Each had previously been screened using the Waterloo-Stanford Group Scale of Susceptibility, form C (Bowers, 1998) with a score of 8 or higher being used to classify them as highly susceptible. This corresponds to the upper 10% of people screened. Scores ranged from 8–11 (*M* = 9, SD = 1.04). Each was paid £5 for participation, the whole experiment lasting approximately 1 h. No participant reported having any type of synesthesia, this was asked prior to testing. The study was granted clearance from the University of Sussex ethics committee.

## **MATERIALS**

The embedded figures stimuli consisted of four shapes (squares, rectangles, triangles and diamonds) of number 2 s embedded in an array of 5 s taken from the Ward et al. (2010) study. Each shape was made from 6 to 10 target 2 s surrounded by 41 distracter 5 s, all of which were in black font. Participants sat approximately 85 cm from a 15<sup>00</sup> LCD monitor with 60 Hz refresh rate. The shape location differed across trials, not always appearing in the center. E-prime 2.0 software was used to run the experiment.

<sup>1</sup> In their original study there were always three kinds of graphemes presented, but the test is commonly illustrated by the example of 5 s and 2 s.

#### **PROCEDURE**

The experiment was repeated twice within each session, counterbalanced so that half of the subjects completed the hypnosis condition first, and the other half the control. Participants were not informed that the study would include hypnosis until just prior to the hypnosis condition to avoid "hold back" (Stam and Spanos, 1980) where participants may unconsciously perform poorer in the baseline condition than they were capable of.

The experiment consisted of four blocks. For half of the participants the block order was ascending (1 to 4 s duration), for the other it was descending (4 to 1 s). In the hypnosis condition, participants received a brief induction (where the participant was asked to become relaxed and counted down into a deep hypnotic state) before a hypnotic suggestion to see green for 2 and red for 5 on the monitor where the digit was, very much like a projector synesthete. Prior to each hypnosis block, the suggestion was reinforced by requiring the participant to focus on an individual stimulus digit and attempt to enhance the color. This was done for both the 2 and 5. If they did not see any color, they were asked to attempt to visualize it as colored as best they could. **Figure 1** shows how the stimulus would be colored if color experiences phenomenologically similar to those of natural projector synesthetes were evoked using hypnotic suggestion. When completing the control block, no specific instructions were given on how to complete the task.

## **SUGGESTION FOR HYPNOTIC GRAPHEME-COLOR SYNESTHESIA**

"Now you will see on the computer screen many 2 s and 5 s. Whenever you see the digit five you will experience it as having a special red color. Similarly, whenever you see the digit two you will experience it as having a special green color. I want you to make the special color as vivid as possible, actually see the color

there. Soon you will be presented with a screen containing both 5's and 2's. You will be able to see the 5's as vividly red and the 2's as vividly green. On each trial there will be a green shape made up of 2's. You must select the shape you see. The trials will be presented for 1, 2, 3 or 4 s. Is that clear? Ok we can start".

An example trial was given at the start of each block where they were reminded to find the shape made of 2 s, and the responses required. Each trial was preceded by a central fixation cross for 1 s. The stimulus was then displayed for 1, 2, 3 or 4 s depending on the block, followed by a blank screen containing instructions to respond using a four-alternative forced choice to indicate the shape (square, rectangle, diamond or triangle). Following this, they were asked to rate their subjective experience of color during the trial display on two scales. Specifically, they were asked to rate the vividness of any perceived colors (1 = no color, 6 = very vivid color) followed by the percentage of digits within the array which were colored. The fixation cross then appeared to signal the start of the next trial, see **Figure 2** for trial sequence. Accuracy was emphasized and participants were aware that the proceeding trial did not begin until a response had been made for the current trial. On completing the hypnosis condition, the suggestion was removed (by stating that numbers no longer had any special colors, and appear as they did before any suggestion was given) then the participant was counted out of hypnosis.

## **RESULTS**

#### **ACCURACY OF TARGET DETECTION**

The accuracy data were measured using percentage accuracy and were analyzed as a 2 × 4 repeated measures ANOVA contrasting condition (presence/absence of hypnotic suggestion) and duration of stimulus (1–4 s). The main effect of duration was significant (*F*(1.78, 22.13) = 45.16, *p* < 0.001) with pairwise comparisons using Bonferroni adjustment showing significant differences between all durations (*p* < 0.001) other than between 3 and 4 s (*p* = 0.13) due to accuracy improving when the arrays were

presented for longer durations (1 s *M* = 46.6%, MSE = 3.0; s *M* = 61.1%, MSE = 4.8; 3 s *M* = 74.0%, MSE = 4.5; 4 s *M* = 79.1%, MSE = 5.3). The main effect of condition was not significant (*F*(1, 13) = 0.62, *p* = 0.45): accuracy for the control condition (*M* = 67.2%, MSE = 5.5) was similar to that in the hypnosis condition (*M* = 63.2%, MSE = 3.9). The interaction was also not significant, with accuracy being similar between control and hypnosis conditions for each duration (*F*(3, 39) = 0.88, *p* = 0.46). This data is depicted in **Figure 3.** To determine whether the lack of main effect of condition reflected insensitive data, or supported a null hypothesis, we used a Bayes factor analysis (Dienes, 2011). Whereas significance testing only allows the null hypothesis to be rejected, Bayes factor analysis also allows the null hypothesis to be supported (Kass and Raftery, 1995). If the Bayes factor is less than 1/3 there is substantial evidence for the null over the specified alternative; if greater than 3, substantial evidence for the alternative; otherwise the data are insensitive in distinguishing the two hypotheses. Ward et al. (2010) found that synesthetes were better than normal on this task by 10%; thus this was used as the standard deviation of a half-normal to represent an alternative hypothesis that the hypnosis suggestion created a genuine synesthesia (as per the guidelines in Dienes, 2011 Appendix). With an actual mean difference of −4% (SE = 5.1%), the Bayes factor is 0.28, i.e., there was substantial evidence for the null hypothesis, that there is no difference between percentage accuracy for the hypnosis vs. control condition.

#### **PHENOMENOLOGICAL REPORTS**

We next considered the extent to which the participants experienced colors during the task. **Table 1** shows the number of trials in which color was perceived, the average vividness of colors reported by each participant, and the percentage of graphemes in the array that were perceived as colored. These data are reported only for the hypnotic suggestion condition. Twelve out of the 14 participants experienced some color during the hypnosis condition. For those who did experience color, the proportion of digits and intensity in which they saw color was extremely variable across participants.

A one-way ANOVA comparing intensity ratings across stimulus duration was not significant (*F*(1.45, 18.87) = 0.19, *p* = 0.76): that is, intensity ratings were similar across all durations (1 s *M* = 2.29, MSE = 0.33; 2 s *M* = 2.41, MSE = 0.39; 3 s *M* = 2.38, MSE = 0.40; 4 s *M* = 2.38, MSE = 0.42). (Note that Ward et al., 2010 reported a similar average vividness rating, 3, for genuine synesthetes.) Similarly, a one-way ANOVA comparing percentage of graphemes perceived as colored across stimulus duration was not significant (*F*(1.52, 19.73) = 0.11, *p* = 0.84) with comparable percentage of grapheme appearing as colored across all durations (1 s *M* = 25.72%, MSE = 7.64; 2 s *M* = 26.25%, MSE = 7.68; 3 s *M* = 24.99%, MSE = 7.72; 4 s *M* = 24.75%, MSE = 7.57) (Ward et al., 2010 reported a similar percentage, 30%, for genuine synesthetes). The duration of stimulus presentation did not substantially affect the phenomenological experience of color.

#### **RELATIONSHIP BETWEEN ACCURACY AND PHENOMENOLOGICAL REPORTS OF COLOR**

In order to better understand the role, if any, that color experience played in shape detection the relationship between accuracy and the phenomenological color reports was explored.

For these analyses, the participants were divided by a median split according to the number of trials in which they reported color experiences thereby creating two groups: many or few color responses to the color suggestion.

There is evidence for a relationship between accuracy and phenomenological report when trials are grouped by accuracy. Participant 10 who never experienced colors had 100% accuracy for the 4 s duration block preventing comparison between correct and incorrect trials; after excluding this participant there remained six participants in the group who experienced few colors and seven in the group who experienced many colors. Using phenomenological ratings (mean intensity, mean number of graphemes perceived as colored) as the dependent variables a 2 × 2 × 4 ANOVA was conducted contrasting group (many vs. few color-responses), accuracy (correct vs. incorrect trials) and duration (4 levels). The data are summarized in **Figure 3**. For intensity, there was a significant interaction of group X accuracy (*F*(1, 11) = 5.09, *p* = 0.045). All other main effects and interactions were not significant. The interaction was analyzed further by simple effects of accuracy for each group. For the group who experienced few colors, the effect of accuracy was not significant (*F*(1, 5) = 0.016, *p* = 0.71) however for the group who experienced many colors this main effect was significant (*F*(1, 6) = 6.32, *p* = 0.046) as more vivid colors were reported for accurate compared to inaccurate trials. This is summarized in **Figure 4**.

For the percentage of graphemes perceived as colored, there was again a significant interaction of group X accuracy (*F*(1, 11) = 5.80, *p* = 0.035). The other main effects and interactions were not significant. A simple effects analysis indicated that for the group who experienced few colors, the effect of accuracy was not significant (*F*(1, 5) = 0.51, *p* = 0.51) whereas for the group who experienced many colors it was significant (*F*(1, 6) = 6.51, *p* = 0.043) with


**Table 1 | Summary of subjective color experiences for each participant for the hypnosis condition only in descending order from the participant who experienced color for the most trials to least**.

The table shows the percentage of trials where a color was reported, average intensity (1 = no color, 6 = vivid color) and percentage of graphemes within the array perceived as colored.

more colors being reported for accurate opposed to inaccurate trials. The data are summarized in **Figure 5**. Together, the results from **Figures 4** and **5** show that participants who saw many colors had more intense and widespread phenomenological color experiences for trials in which they correctly compared to incorrectly identified the embedded shape.

Although the different groups report different levels of color intensity and disparity on correct vs. incorrect trials, the overall number of correct trials didn't differ according to these groups. A 2 × 4 repeated measures ANOVA with percentage correct as the dependent variable contrasting group (experiencing few vs. many colors) and duration of stimulus (1–4 s) was conducted, considering only the hypnotic-suggestion condition. This data is summarized in **Figure 6.** The main effect of duration was significant (*F*(1.67, 20.09) = 18.88, *p* < 0.001) with accuracy improving as stimulus duration increased. The interaction between group and stimulus durations was not significant (*F*(1.67, 20.09) = 0.31, *p* = 0.70). Importantly, the difference in accuracy for the hypnosis block between those who experienced many (*M* = 61.7%, MSE = 5.7) and few (*M* = 64.7%, MSE = 5.7) color responses to the synesthesia suggestion was not significant (*F*(1, 12) = 0.13, *p* = 0.72). To interpret the non-significant result, a Bayes Factor analysis was conducted. Again a half-normal distribution was chosen to test the alternative hypothesis that the group who experienced many vs. few colors performed better, representing a real synesthesia like behavioral advantage in those who had phenomenological experience of the colors. As Ward et al. (2010) measured a 10% accuracy advantage for synesthetes, this was used as the SD. With a mean difference of −3, and MSE of 9.5 the resulting Bayes Factor was 0.58 which is between 1/3 and 3 and therefore indicates insensitive data.

## **DISCUSSION**

We aimed to determine whether hypnotic synesthesia was similar either behaviorally or phenomenologically to developmental synesthesia through measuring accuracy and color experience during an embedded figures task with and without hypnotic suggestion. Under the hypnotic suggestion the phenomenology of the participants was similar to that documented for synesthetes. Specifically, they tended not to perceive the entire array of graphemes as colored and the subjective intensity ratings were similar to those reported by synesthetes. However, the behavioral advantage previously found for synesthetes was not found under hypnotic suggestions, even when one only considers those subjects who responded strongly to the specific suggestions. Further analyses (using Bayes factors) suggested that this was not merely due to a lack of sensitivity. As such, our conclusion is that hypnotically induced grapheme-color experiences are not equivalent to those in developmental synesthesia.

On first impressions, this result seems at odds with Cohen Kadosh et al. (2009), who found that hypnotically suggested synesthesia results in similar performance to developmental synesthetes, showing a deterioration in the ability to detect an achromatic grapheme when the concurrent matched the background color. However, because this study predicted an impairment (as opposed to an enhancement), their task is potentially more susceptible to demand characteristics. We note that the experimenter in the present study was not blind to the experimental condition. However, the principal effect (if any) of lack of blindness would be to amplify demand characteristics, which we have argued are less likely to apply in our study than in Cohen Kadosh et al. (2009). The combination of a strong behavioral effect in Cohen Kadosh et al., and none at all in our study, is most simply explained by subjects responding according to how they believe they should, without hypnotically-induced alterations in perceptual abilities. This claim is entirely consistent with subjects in both studies having subjectively compelling experiences.

It is important to note that the ability to respond to the synesthesia suggestion was very variable across our participants. This is perhaps not surprising since perceptual hallucinations are difficult to evoke even in highly hypnotizable subjects (Bowers, 1998). It should also be noted that many developmental grapheme-color

synesthetes fail to report colors during this task, at least during brief (1 s) presentations of the array (Ward et al., 2010). The strength of subjective experience of the colors was comparable to that of synesthetes. Anecdotally, some of our participants noted that the hypnotically suggested colors appeared to diminish over time. To reduce the impact of this, the suggestion was reinforced between blocks to sustain the colors but several participants struggled to maintain the suggestion all the same. Future research should combine extensive training of grapheme-color pairings (e.g., Rothen et al., 2011) with subsequent hypnotic suggestion.

If (at least some) of our hypnotized participants reported color experiences then why didn't this help them to detect the embedded figure? One possibility suggested by our results is that the color hallucinations are primarily added after grapheme (and global figure) identification or, relatedly, that the "task" of adding color visualizations competes with the primary task of finding the embedded figure. This is supported by the analysis in which participants were divided (by median split) according to

whether they reported many or few color visualizations. For the half of participants who experienced many colors, significantly more vivid colors were reported for accurate compared to inaccurate trials. This difference was not evident in the group who did not experience much color, perhaps due to a floor effect given that so little color was reported by these participants. The enhanced color experience for correct compared to incorrect trials for those who did experience color may reflect the ease with which the colors could be projected onto the graphemes by the participant. In this view, once the shape has been detected, the identity of digits within the array can more easily be inferred, potentially allowing easier visualization of the spatial localization of the red and green colors. In trials in which the shape has not been detected, the participant is performing two tasks at once; the conscious task of identifying the shape and the "unconscious" task of projecting colors onto black graphemes. The process of binding the grapheme with the concurrent color does not seem to occur as automatically in hypnotically suggested synesthesia, as compared to developmental synesthesia. Indeed, there is evidence that many hypnotic responses take up capacity by virtue of being hypnotic (Hilgard, 1986; Tobis and Kihlstrom, 2010; Wyzenbeek and Bryant, 2012; contrast Woody and Bowers, 1994). Developmental and artificially induced variants of synesthesia (i.e., hypnotic or drug induced) may be different. Auvray and Farina (in press) have explored this issue, and using their characterization of developmental synesthesia (as satisfying the criteria of the pairing of an inducer with a conscious concurrent, the idiosyncratic nature of the concurrent, and the concurrent being produced automatically and consistently) they have suggested hypnotic synesthesia satisfies the requirements of having a concurrent paired with an inducer, in an idiosyncratic and automatic way, but that consistency requires further investigation. Further, they suggest that the concurrent may be produced by imagery. Our results support a limited similarity between developmental and hypnotic synesthesia, and showing that despite the phenomenology, the concurrent may not be automatically produced (as shown by a lack of behavioral improvement).

How is it possible that our participants were able to generate color experiences at all (assuming, that is, that their subjective reports had some basis)? One possibility is that it relies on mechanisms normally used to support visual imagery. However, we did not assess this directly in our research. Other research suggests that there are individual differences between high and low hypnotizable subjects in the tendency to employ imagery in suitable contexts (e.g., Tellegen and Atkinson, 1974; Hilgard, 1979; Wilson and Barber, 1982; Roche and McConkey, 1990; Lynn and Green, 2011). However, the tendency to employ imagery in certain contexts may reflect strategic differences rather than ability differences given that high susceptible participants are not especially quick at visual information processing (Acosta and Crawford, 1985; Friedman et al., 1987) and are not especially high in rated imagery vividness (Jamieson and Sheehan, 2002; though compare the feats of imagery achieved by high but not low susceptible subjects in Mazzoni et al., 2009). These suggestions are tentative given that we did not run low hypnotizable subjects. Further, measuring the mental imagery abilities of participants would help clarify to what extent participants are able to use mental imagery to complete the tasks and how this relates to the individual profiles of hypnotic suggestibility (Cardeña, 2005; Terhune and Cardeña, 2010).

The lack of behavioral advantage for hypnotic synesthetes can be taken as evidence against functional similarity between hypnotic and natural synesthesia, but by the same token it provides support for the Cold Control theory of hypnosis (Dienes and Perner, 2007), and the class of theories which postulate no special ability is gained when an action is performed hypnotically (e.g., Kirsch and Lynn, 1999). Cold Control theory states that the subjective lack of volition in hypnosis is due to not forming the higher order thought (HOT) linked to the intention. In this sense, someone who responds to the suggestion "lift your arm" could lift their arm but not have the HOT "I am lifting my arm". If this theory holds, then participants should not be able to perform better in the hypnosis block then they do in the control block. Our data indeed support this inference. Theories that postulate that hypnotic hallucination is perception-like in a way that goes beyond normal imagery (e.g., Brown and Oakley, 2004) are challenged by the current results.

In summary, hypnosis can induce verbal reports of phenomenological experience of grapheme-color synesthesia similar to those provided by developmental grapheme-color synesthetes, when applied in high susceptible participants. However, even though there are strong similarities in the subjective reports of natural and hypnotic synesthetes, this in not reflected in behavioral similarities. Highly hypnotizable subjects do not gain any perpetual abilities with a hypnotic suggestion that they did not have prior to hypnotic induction (Dienes and Perner, 2007).

## **AUTHOR CONTRIBUTIONS**

All authors (Hazel P. Anderson, Anil K. Seth, Zoltan Dienes, and Jamie Ward) contributed to the experiment design, analysis and write up. Hazel Anderson conducted the study.

## **ACKNOWLEDGMENTS**

We are grateful for support from Mr. Jonathan Laredo and from the Dr. Mortimer and Dame Theresa Sackler Foundation.

## **REFERENCES**


**Conflict of Interest Statement**: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 31 October 2013; accepted: 28 March 2014; published online: 28 April 2014*.

*Citation: Anderson HP, Seth AK, Dienes Z and Ward J (2014) Can grapheme-color synesthesia be induced by hypnosis? Front. Hum. Neurosci. 8:220. doi: 10.3389/fnhum. 2014.00220*

*This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2014 Anderson, Seth, Dienes and Ward. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms*.

## A longitudinal study of grapheme-color synesthesia in childhood: 6/7 years to 10/11 years

## *Julia Simner\* and Angela E. Bain*

*Department of Psychology, University of Edinburgh, Edinburgh, UK*

#### *Edited by:*

*Nicolas Rothen, University of Sussex, UK*

#### *Reviewed by:*

*Beat Meier, University of Bern, Switzerland Lawrence E. Marks, Yale University, USA*

#### *\*Correspondence: Julia Simner, Department of Psychology, University of Edinburgh, 7 George Square, Edinburgh, EH8 9JZ, UK e-mail: jsimner@ed.ac.uk*

Grapheme-color synesthesia is a condition characterized by enduring and consistent associations between letter/digits and colors. This study is the continuation of longitudinal research begun by Simner et al. (2009) which aimed to explore the development of this condition in real time within a childhood population. In that earlier study we randomly sampled over 600 children and tested them aged 6/7 and 7/8 years. We identified the child synesthetes within that cohort and measured their development over 1 year, in comparison to a group of non-synesthetic children with both average and superior memories. We were able to show the beginnings of a developmental progression in which synesthetic associations (e.g., *A* = red) mature over time from relatively chaotic pairings into a system of fixed consistent associations. In the current study we return to this same population three years later when participants are now 10/11 years. We used the same paired-association memory task to determine the synesthetic status of our participants and to also establish synesthetes' inventories of grapheme-color associations. We compared their inventories to those from age 6/7 and 7/8 years to examine how synesthesia matures over time. Together with earlier findings, our study shows that grapheme-color synesthesia emerges with a protracted trajectory, with 34% of letters/digits fixed at age 6/7 years, 48% fixed at 7/8 years and 71% fixed at 10/11 years. We also show several cases where synesthesia is not developing in the same time-frame as peers, either because it has died out at an older age, or because it was slower to develop than other cases. Our study paints the first picture of the emergence of synesthesia in real-time over four years within a randomly sampled population of child synesthetes.

#### **Keywords: grapheme-color synesthesia, development, consistency, children, longitudinal, childhood**

## **INTRODUCTION**

For people with grapheme-color synesthesia, letters or digits have fixed enduring conscious color associations. For example, the letter A might be red, B might be blue, C might be yellow and so on. In the terminology of the literature, letters and digits are the "inducers" of grapheme-color synesthesia, and the color itself (i.e., the synesthetic experience) is the "concurrent" (Grossenbacher, 1997). Synesthetes report that concurrent colors are either experienced in the mind's eye, or can be projected out into space onto objects such as the written typeface (Dixon et al., 2004). Studies have established that grapheme-color associations have a likely genetic aetiology (Asher et al., 2009; Tomson et al., 2011), that they are neurologically driven (e.g., Hubbard et al., 2005; Rouw and Scholte, 2007; Weiss et al., 2009) and that they tend to be automatically triggered without conscious effort on the part of the synesthete (e.g., Dixon et al., 2000). Despite a relative wealth of contemporary findings on synesthesia produced in recent years, questions about the *development* of synesthesia have been largely overlooked in empirical terms. This study aims to address this by examining how these unusual associations are acquired throughout childhood in a random sample of child grapheme-color synesthetes.

Within the field of synesthesia research, the study of development in childhood has been hindered by a difficulty in recruiting child synesthetes. Other neurodevelopmental child cohorts (e.g., children with autism) can be readily found following their referrals to clinics, but these routes are not available for researchers of synesthesia since the condition is not classified as pathology. Options for recruitment are therefore limited to inviting participation from the synesthetic children of known adult synesthetes (Green and Goswami, 2008), or to the time-intensive scanning of large random populations to detect the comparatively small number of child synesthetes within them (Simner et al., 2009). The former approach raises questions of scientific validity because the children of known adult synesthetes, often study participants themselves, are unlikely to represent the population of child-synesthetes at large. Instead they are the children of family environments led by motivated parents with an interest in scientific research and the engagement in science, and in particular with an interest in synesthesia (Simner et al., 2009). For all these reasons both the children themselves and the children's synesthesia may be non-representative of what we might expect from the average synesthete child, randomly sampled. The only study to have attempted the latter screening approach in order to find randomly sampled child synesthetes (Simner et al., 2009) was considerably time-intensive because the rarity of synesthesia requires screening large samples for comparatively small numbers of synesthetes (e.g., 8 grapheme-color synesthetes found in 615 children screened by Simner et al., 2009). However, once found, those child synesthetes were, and remain, an extremely valuable population, and we know of no other similar cohort anywhere in the world. For this reason, we continue to explore the development of these children in the current paper, showing how their condition has matured now they are four years older than when first tested.

The requirements of our current study—and those of Simner et al. (2009)—were first to have an objective means of identifying child synesthetes from a large sample of children, and second to have a way to evaluate how their synesthesia is developing over time. The test to identify synesthetes used by Simner et al. (2009), and again here, is based on the behavioral "gold standard" task (Rich et al., 2005): a test of *consistency over time*. In this test, synesthetes are identified by the fact that people with synesthesia tend to have consistent and enduring inducer-concurrent relationships. In other words, for any given synesthete, each inducer (e.g., a letter) will tend to elicit the same concurrent color over time (e.g., if the letter A is red, it tends to always be red for that particular synesthete, in repeated testing). This consistency was first demonstrated by Baron-Cohen et al. (1987) who showed that a grapheme-color synesthete reported the same colors for over 100 letters and words in two different testing sessions separated by more than two months. This consistency effect is perhaps the most replicated finding in the synesthesia literature: studies have replicated it in grapheme-color synesthesia (Dixon et al., 2000; Edquist et al., 2006; Eagleman et al., 2007), sound-color synesthesia (Ward et al., 2006), word-taste synesthesia (Ward and Simner, 2003), and so on—indeed, within virtually every study of synesthesia within the last decade. Across these studies, synesthetes tend to be between 80 and 100% internally consistent over time (e.g., Ward et al., 2005); in contrast, a non-synesthete control tends to be around 25% internally consistent when asked to generate and then recall analogous associations (e.g., to freeassociate colors to letters and then recall these associations in a retest). Indeed, controls score poorly even when tested over a considerably shorter time interval (e.g., 17% consistent when retested after just 2 weeks; Baron-Cohen et al., 1987). Given this, the test of consistency has been adopted as the primary means for identifying genuine synesthesia (but see Simner, 2012 for discussion): synesthetes are required to be significantly more consistent in recalling their associations compared to a group of non-synesthete controls.

The consistency test used to identify child synesthetes in Simner et al. (2009) will be used again in the current study. Simner et al. tested 615 children aged either 6 or 7 years (which we henceforth refer to as "aged 6/7 years"). Each child took part in an on-screen test in which they viewed the letters of the alphabet and the digits 0–9, one by one, in a random order. Accompanying each grapheme was an on-screen palette of 13 colors, and children were required to pair each grapheme with a color as they saw fit. When all graphemes had been shown, the test paused for approximately 10 s, then participants repeated the test again. Simner et al. compared the grapheme-color choices made before vs. after the 10-s pause, to establish for each child how consistent his/her color choices had been across that short interval. On average, children selected the same color for only 3.5 out of the 36 graphemes. However, 47 children had consistency scores significantly higher than the mean and these were classified as "potential synesthetes"—in that their high consistency may have been indicating they were synesthetes. To further clarify this, Simner et al. returned one year later, when the children were now aged 7/8 years. Simner et al. presented the same test again, but this time compared the grapheme-color associations made by each "potential synesthete" across the entire year that had passed. By doing this they found eight children who continued to be highly consistent even across this extensive time interval. These eight children were therefore categorized as genuine synesthetes on the assumption that maintaining this consistency would be a hugely challenging task for non-synesthetes. Indeed, Simner et al. required that their synesthetes had consistent colors both across 12 months (compared to their entire peer group over just 10 s, in Session 1) and also across 10 s at age 7/8 years (and here they were compared to 40 orthogonally age/sex-matched controls who were also selected for retesting at age 7/8, because they had been representative of perfectly average non-synesthete children in Session 1). After removing these eight "genuine synesthetes," Simner et al. classified the remaining 39 "potential synesthetes" as "high-memory non-synesthetes," since these children had likely performed well when first tested (i.e., over 10-s) based solely on a high memory span.

In the current study we aimed to repeat this process with the same cohort of children, now that they are aged 10/11 years (i.e., 8 synesthetes, 39 high-memory non-synesthetes and 40 average-memory non-synesthetes). Our first intention was to establish whether the child synesthetes identified previously are still exhibiting the behavior of synesthetes after three more years have passed. It is possible that synesthesia may die out in some individuals, and there are three reasons to suspect this. First, there are anecdotal reports from adults suggesting they had synesthesia in childhood which they no longer have. Second, Simner et al. (2009) found a numeric trend toward finding more synesthetes in their younger (originally aged 6) vs. older (originally aged 7) age group, although the small numbers in that study gave insufficient power to test this statistically. Finally, a decline in synesthesia through childhood is also predicted by the most widely accepted developmental model of synesthesia: the *Neonatal Synesthesia Hypothesis* (Maurer, 1997). This model proposes that all neonates and young infants experience sensory/perceptual stimuli in a synesthetic manner (Spector and Maurer, 2009). However, children are thought to gradually lose these exaggerated abilities through selective pruning of synaptic connections in childhood because these connections may no longer be adaptive to development and general cognition (Spector and Maurer, 2009). Synesthetes, on the other hand, are thought to maintain these connections into adulthood, either via absent or delayed synaptic pruning. These proposals are backed up by infant neuroimaging studies: verbal stimuli can activate both verbal and visual cortex in a way that is not observed in the adult population (Neville, 1995) thereby suggesting a less modularized processing in infants. Additionally, sound and color associations have also been found in infants that mirror those observed later in older sound-color synesthetes (Mondloch and Maurer, 2004). If the neonatal synesthesia hypothesis is correct, this would suggest that synesthesia may die out in some children as the normal pruning processes in development run their course (see Huttenlocher and Dabholkar, 1997 for a discussion on the time-course of this pruning). Our study will test whether any children previously identified as synesthetes fall below that classification now that more time has passed.

As well as identifying child synesthetes, Simner et al. were able to evaluate how these children's synesthesia had developed over the one year that had passed between their two testing sessions (i.e., between Session 1 when children were 6/7 years, and Session 2 when the children were 7/8 years). Simner et al. found that on average, genuine synesthetes acquired 6.4 new fixed graphemecolor associations over that year: at age 6/7 years synesthetes were consistent over 10 s in the coloring of 10.5 graphemes on average (out of 36 letters and numbers), while at age 7/8 years they were consistent on 16.9 graphemes over 10 s. In other words, synesthetes were moving from a state where their colors were largely in flux, to a state where almost half of graphemes were fixed to a consistent color, at least over a 10-s retest. In the current study we will test how this pattern of acquisition has continued to develop now the children are aged 10/11 years. Since adults have consistent colors for around 80–100% of their graphemes, we anticipate a higher proportion of fixed colors in older vs. younger children. We will assess this by comparing how consistently our synesthetes pair graphemes with colors over a 10 s retest at 10/11 years to how they did when they were younger (at age 6/7 and /7/8 years).

In summary, Simner et al. (2009) tested children aged 6/7 years (**Session 1**) and again aged 7/8 years (**Session 2**). Children made grapheme-color selections twice in each session and we label these **Selections 1a/b** and **Selections 2a/b**. In the current study, we will test the children again in a **Session 3**, using the same task where they will again make grapheme-color selections twice across a 10 s gap (**Selections 3a/b**). We will again identify synesthetes as those children who show consistent color choices both in **immediate retest** (Selection 3a vs. 3b; i.e., a 10 s gap) and in a **delayed retest** (Selection 3 vs. Selection 1; i.e., a 4-year gap<sup>1</sup> ). As before synesthetes will need to significantly outperform their peers on both criteria. Once synesthetes are identified here we will next evaluate how many of their graphemes now have fixed colors at age 10/11 years (Selection 3a vs. 3b) and compare this to how many were fixed at age 7/8 years (Selection 2a vs. 2b) and at age 6/7 years (Selection 1a vs. 1b). We predict that synesthetes will show an incremental increase in the number of fixed colors from the younger to the older testing age.

#### **EXPERIMENTAL INVESTIGATION**

The eight previously identified synesthetes and their high and average memory controls will be given the same consistency task administered in previous testing sessions by Simner et al. (2009) and will be assessed using similar criteria. The synesthetes that continue to consistently associate graphemes and colors to a significant degree, in both immediate and delayed retests, shall be re-confirmed as genuine synesthetes at 10/11 years old. The development of grapheme-color synesthesia shall then be investigated by assessing the acquisition of synesthetic associations from ages 6/7 to 10/11 in comparison to the development of non-synesthetic children with both high and average memories.

#### **METHODS**

#### *Participants*

We tested 80 of the 87 children previously tested by Simner et al. (2009). In that earlier study, children had been identified as either synesthete, high-memory non-synesthete or averagememory non-synesthete (see above). Seven high-memory and 10 average-memory non-synesthetes were untraceable for the current study, although we were able to replace all 10 averagememory non-synesthetes with children from the same population (i.e., children whose immediate consistency score in Session 1 fell within the average range: either 3 or 4, out of 36 graphemes). The original 40 average-memory non-synesthetes tested by Simner et al. had been orthogonally crossed for age (6 vs. 7 years in Session 1) and sex; our replacements allowed us to fully maintain this for age, and virtually maintain this for sex (19 males; 21 females). We also tested the eight children previously classified as synesthetes (five female) and the remaining 32 classified as highmemory non-synesthetes (22 females). All participants were 6/7 years when first tested by Simner et al. (2009), and are described here as aged 10/11 years for conciseness, although five children were still 9 at the time of our testing here, and ten children had just turned 12 (due to the particular timings and duration of our testing). Our study was approved by the local ethics board and written consent was given by head teachers and parents.

#### *Materials and procedure*

Our consistency task was a computerized test taken from Simner et al. (2009; and previously, from Simner et al., 2006) which individually presents the 26 letters and the digits 0–9 (i.e., 36 graphemes) alongside an electronic palette of 13 colors (black, dark blue, brown, dark green, gray, pink, purple, orange, red, white, light blue, light green and yellow). These colors are based on Berlin and Kay's (1969) irreducible color terms, with the addition of dark and light variants of blue and green<sup>2</sup> . Both the order of graphemes and the trial-by-trial configuration of colors were randomized for each participant. Graphemes were displayed in black Ariel font against a white background on the left side of the screen, taking up approximately 40% of the screen-height. As in Simner et al. (2009) participants were told they would

<sup>1</sup>See methods for how delayed consistency is calculated with respect to the four different comparisons possible here (Selections 3a vs. 1a; 3b vs. 1b; 3a vs. 1b; 3b vs. 1a).

<sup>2</sup>Simner et al. (2009; p. 59) point out "[a]lthough synesthetes" colors are highly specific (e.g. Ward et al., 2005) and could be closely approximated with sufficient time and effort using an extensive palette comprising many thousands of colors (as we have done elsewhere; e.g., Simner et al., 2006), prior studies have shown that a presentation of our condensed palette allows for a successful assessment of synesthesia that is practical for large numbers of participants, or for those with limited attention (Day, 2005; Simner et al., 2006). When faced with the core palette, synesthetes are systematic in their choices, while non-synesthetes are significantly more random (Simner et al., 2006)."

be playing a short computer "game" and would be required to select with a computer mouse the "best" color for each grapheme as promptly as possible. They were also told that there was no right or wrong answer but that they should avoid selecting the same color repeatedly. A paper print-out of a screen-shot was used to illustrate what the task entailed. Approximately 10 s after completing all 36 graphemes, participants performed an immediate surprise retest, in which the order of graphemes and color configurations were re-randomized. (Specifically, our program presented all 36 graphemes in a random order, then paused for 10 s, then began again, showing the same, but re-randomized, graphemes. During the pause, children were told to wait a few moments and then to "carry on as before.") The task took ∼5 min to complete.

#### **RESULTS**

#### *Synesthete status*

We remind the reader that our participants have now been tested in three sessions, and that within each session they made two color selections for each grapheme, either side of a 10-s pause. We have termed these: Session 1 (age 6/7 years; **Selections 1a and 1b**), Session 2 (age 7/8 years; **Selections 2a and 2b**) and Session 3 (age 10/11 years; **Selections 3a and 3b**). Sessions 1 and 2 were conducted by Simner et al. (2009) and Session 3 is added here.

Below, we identify children who are "Session 3 synesthetes (aged 10/11 years)" by comparing how consistently our participants paired graphemes to colors both within and across testing sessions. Following Simner et al. (2009), for a child to qualify as a synesthete he/she must satisfy both of the following requirements: be significantly more consistent in *immediate consistency* (Selections 3a vs. 3b) compared to average-memory non-synesthetes AND be significantly more consistent in *delayed consistency* over four years (Selections 1a vs. 3a *or* Selections 1b vs. 3b <sup>3</sup> ) compared to the mean consistency of their peer-group (*n* = 615) over 10 s in Session 1. (To rephrase this, *delayed consistency* requires that the associations made by synesthetes at age 6/7 years remain significantly more consistent over four years, than their peers' associations remained over 10 s only).

We shall assess all 80 children for synesthesia, notwithstanding the fact that only eight were recognized as synesthetes at an earlier age by Simner et al. (2009). This is because some true synesthetes may have evaded identification in Sessions 1 and 2, perhaps because their development was slower than their synesthetic peers. (Equally, we may also find that some cases of verified synesthesia may now have declined with age).

#### *Immediate consistency*

To achieve the status of a genuine synesthete at 10/11 years, a child must first significantly outperform average memory nonsynesthetes (*n* = 40) tested for immediate consistency in Session 3. Participants' scores were collapsed across both age and sex since Session 3 average-memory controls showed no significant effect for either factor (although there was a trend for girls to perform better) [age: *Fage(*1*,* <sup>36</sup>*) <* 1; *Fsex(*1*,* <sup>36</sup>*)* = 3*.*31, *p* = 0*.*08] and there was no interaction [*Fage*∗*sex(*1*,* <sup>36</sup>*) >* 1]. We calculated a cut-off value at 1.96 standard deviations above the control mean, both for letters (mean = 4.1; *SD* = 2*.*9; cut-off = 9.8) and numbers (mean = 1.7; *SD* = 1*.*6; cut-off = 4.9). As such, synesthetes had to achieve a consistency of at least 10/26 for letters or 5/10 for numbers. We describe how participants satisfied these criteria further below.

#### *Delayed consistency*

The delayed consistency criterion required synesthetes to score significantly higher over a four years gap than their peer group (*n* = 615) had scored over 10 s in Session 1. In Session 1, girls had performed similarly to boys although 7 year olds (*n* = 275) significantly outperformed 6 year olds (*n* = 332). As such our delayed consistency analysis in Session 3 requires us to assess the two age groups separately. **Table 1** summarizes the means, standard deviations and synesthetic criteria (1.96 SD above the mean) for each age group, from Simner et al. (2009); synesthetes who were 6 when first tested now require a 4-year delayed consistency of at least 6/26 for letters or 3/10 for numbers, while those who were 7 years require 8/26 or 4/10.

Six children indeed satisfied both immediate and delayed consistency criteria, and on this basis can be recognized as Session 3 synesthetes (aged 10/11 years). These were five of the eight synesthetes from Simner et al. (2009; HM, JC, DJ, GT, MA) and one child previously categorized as a high-memory non-synesthete (CM; female; aged 6 when first tested). All children who continued to be classified as such from in Simner et al. (2009) remained within the same variant(s) (i.e., either letter-color and/or digitcolor) apart from DJ who moved from letter/digit-color synesthesia to just letter-color synesthesia.

We also identified six additional children who were very close to achieving our rigorous standards, three of whom were previously identified by Simner et al. (2009) as synesthetes



*Table shows means and standard deviations (SDs) from immediate consistency over 10 s in Session 1 (i.e., Selection 1a vs. 1b) for letters and digits. It also shows the values of the "cut-offs" by which we identified synesthetes (mean plus 1.96\*SD; i.e., synesthetes must perform significantly higher than the mean). Session 3 synesthetes must score at or above this cut-off in a four-year retest.*

<sup>3</sup>Following Simner et al. (2009), we point out that children gave four colors for each grapheme in total in Sessions 1 and 3. Therefore, delayed consistency could, in theory, be calculated by any of four different comparisons (Selections 1a vs. 3a; 1b vs. 3b; 1a vs. 3b; 1b vs. 3a). As in Simner et al. (2009), we conservatively discounted matches across first and second presentations (e.g., Selections 1a vs. 2b), and conservatively disallowed high consistency to be achieved by a mix of consistency from within first presentations and within second presentations (e.g., four consistently cultured graphemes in Selections 1a vs. 2a, and three consistently cultured graphemes in Selections 1b vs. 2b does not equal a score of seven). For further conservativeness, we discounted any matches across light and dark versions of the same color (e.g., "light green" vs. "dark green" was considered a mismatch).


**Table 2 | Session 3 synesthetes and "near-misses," indicated by checks and question marks respectively.**

*Columns 1–3 show ID (with previous status in sub-script; s, synesthete, h, highmemory non-synesthete, a, average-memory non-synesthete), sex (m, male; f, female), and age in years when first tested by Simner et al. (2009). Columns 4–5 show immediate consistency in Session 3 (Selections 3a vs. 3b), out of 26 for letters, and 10 for digits. Gray highlighting indicates that this immediate consistency satisfied the synesthetic criterion.*

(CC, CD, DM), two as high memory non-synesthetes (MW, ET), and one as an average memory control (MH; whom we recruited as a replacement so did not test in Session 2). These six children achieved the required consistency either for immediate or delayed consistency, and missed the other criterion by only one point (i.e., fell short on only one letter or digit). In all cases, we verified that high consistency had not been achieved by using only a limited color palette (e.g., choosing the color red throughout) and indeed, on this basis, we eliminated a further child who had also approached significance.

In **Table 2** above we summaries our findings from Session 3 for all children of interest described above. Our six confirmed Session 3 synesthetes are above the triple line, and our "near-misses" (i.e., those who satisfied one criterion and almost satisfied the other) are below the line. Their statuses are shown by checks or question marks, respectively, in the final two columns. The previous status of all children at a younger age (from Simner et al., 2009) is shown by sub-script next to their ID. The table also shows immediate consistency scores for letters and digits in Session 3, with gray shading indicating cells where immediate consistency was within the synesthetic range. Note therefore, that any children considered "near misses" who do *not* have gray shading (i.e., DM and ET) satisfied synesthetic consistency in delayed, rather than immediate consistency.

An example of the striking consistency of synesthetes across all three testing sessions can be seen in **Figure 1**. This shows the colored digits of synesthete JC when she was 6 years old, and then

again one, and four years later. It is clear that the selections she made in Session 1 remain considerably more consistent over four years than those made by an age-matched average control (SB) over just 10-s.

#### *Synesthetic development*

Above we have identified six Session 3 synesthetes (age 10/11 years). Here we now consider the development of their synesthesia over time. By examining their immediate consistency scores in Session 3 (i.e., Selections 3a vs. 3b) we can estimate how many graphemes had their own single fixed color at this age, at least to the extent that this color remained consistent in a 10-s retest. Our approach shows that on average, 10/11 years old synesthetes had fixed colors for 25.7 out of 36 graphemes (SD 4.1), while high memory non-synesthetes had 7.4 (SD 4.4), and average non-synesthetes had 5.8 (SD 4.2). To evaluate these group differences we have remained agnostic about the status of the six "near-miss" children who fell just short of synesthesia (see above). Instead, we have left these six within their original groups (i.e., either high or average memory non-synesthetes) apart from CC, CD, and DM whom we have moved from the synesthete to high memory group (since they had superior immediate consistency in session 1 but are not currently considered synesthetes).

**Figure 2** shows our Session 3 data combined with earlier sessions from Simner et al. (2009), to illustrate how synesthetes have developed fixed colors over time. **Figure 2** shows the immediate consistency for 36 colored graphemes (over 10-s) at ages of 6/7 years (Selections 1a vs. 1b) and 7/8 years (Selections 2a vs. 2b) and 10/11 years (Selections 3a vs. 3b). We have converted these consistencies to percentages to better illustrate the rate of development.

**Figure 2** shows that synesthetes have a sharper rate of acquisition of colored graphemes, compared to what might be expected from other children with average or high-memories. Over the 4-year testing period, synesthetes acquired on average 13.3 new fixed colored graphemes (9.2 letters, and 4.0 digits, within each variant), while high and average memory non-synesthetes improved across sessions by less than 3 graphemes.

We point out that that a subset of our synesthetes achieved synesthetic status for letters only (rather than for all graphemes) meaning that they should not be expected to achieve 36 fixed colored graphemes even in a fully-formed adult state. However, we can report that the pattern of acquisition shown in **Figure 2** is a

place at these ages.

good approximation for the pattern when synesthetes are broken down by type. For example, **Figure 2** shows approximately 71.3% of the 36 graphemes were fixed in color for all synesthetes at aged 10/11 years, and indeed this figure is very similar for those with the letter-color variant (70.5%) or the digit-color variant (73.5%). Nonetheless, for further clarification on how synesthesia develops within each variant, **Figure 3** below shows the case-by-case development of letter-color synesthetes (left panel) and digit-color synesthetes (right panel) separately. We have included not only children classified as synesthetes in both Session 3 and Session 2 (indicated by solid lines), but also those classified in Session 2 only (indicated by dotted lines) and those classified in Session 3 only (indicated by gray, rather than black, lines). These latter two groups can be thought of as children having synesthesia that is, respectively, either dying out, or developing more slowly than their peers. Finally, we also show in **Figure 3** the progression of average memory controls (dash/dotted line with diamond data-markers).

## **DISCUSSION**

Our study re-assessed the synesthetic status of children aged 10/11 years who had previously been identified as synesthetes by Simner et al. (2009) when they were aged 6/7 and 7/8 years. We again identified synesthetes using the gold standard behavioral measure assessing how consistently graphemes are paired with colors over time. In our test, children aged 10/11 years paired graphemes with colors, and their selections were compared across a 10-s retest (*immediate consistency*), and also to selections they had made four years earlier (*delayed consistency*). Like Simner et al. (2009) we required synesthetes to outperform controls in both types of measure. We also assessed the synesthetic status of a group of children who had previously been classified as being non-synesthetes with high memories, on the assumption that some may in fact be more-slowly developing synesthetes. Finally, we also tested here a group of average memory non-synesthetes to serve in a control condition. Once we had identified who were synesthetes from among our total sample, we compared their inventories of colored graphemes at different ages to observe how their synesthesia had developed over time (from age 6/7 to 7/8 to 10/11 years).

We found that five of the eight children previously identified as grapheme-color synesthetes at a younger age continued to satisfy synesthetic criteria in our current study. In other words, these children have now shown hallmarks of synesthesia across four years. We also found a sixth synesthete, not previously classified as such in earlier testing. Her case is unusual: her colors were consistent over 10 s when she was 6, then over one year, and again over 4 years, and they were also highly consistent over 10 s within this final test session (aged 10/11). However, in Session 2 there was a lot of variation in her colors when immediately retested, and for this reason she was classified by Simner et al. (2009) as a high-memory non-synesthete. If our current assumption is correct, and she is indeed a genuine synesthete, we have two possible interpretations: either there was a period of particular flux in her synesthetic colors when she was 7 years old (in Session 2) or she was tired/distracted during that test. The latter is at least plausible, since her first selection of colors in Session 2 (i.e., Selection 2a) were significantly consistent with those from a younger age, while her second selection (Selection 2b) was not (0% match). It is possible therefore that her attention to the task waned during the second testing session conducted by Simner et al. (2009).

Our study might also lend support to the notion that synesthetic ability can die out over time, because three children who were previously classified as synesthetes at an earlier age no longer satisfied these requirements in our most recent test. Since each evaluation of synesthesia is made against age-matched controls, we can assume these children *were* synesthetes at an earlier testing session, and hence, we might therefore have empirical evidence of what could be considered "synesthetic demise" (i.e., the decline of synesthesia within an individual such that it is no longer apparent at a recognized level at a later age). This seems to fit with previous anecdotal evidence from adults who have given subjective accounts of synesthetic ability in childhood that they no longer possess as adults. This is also evidenced by the fact that one of our Session 3 synesthetes has gone through a transition from being a letter and digit synesthete at age 7/8 to only a letter synesthete at age 10/11. Nonetheless, it is possible too that the synesthesia of these three children is merely different in characteristic to those of their peers. Two of these cases in particular were highly consistent over 10 s when tested here, but failed to show high enough consistency compared to the colors they provided four years ago. It is therefore possible that they remain synesthetes but that their colors have simply developed away from those first experienced when they were much younger. If this is true, we would predict that a future test of consistency will show high consistency between their current inventories (age 10/11), and those generated at a future date.

For our unequivocal synesthetes, we also examined their development across four years and found that grapheme-color synesthesia appears to moves from a more chaotic to less chaotic system. When synesthetes were 6/7 years old, only around a third of graphemes on average had colors fixed enough to endure in a 10-s retest. By the age of 7/8 this had risen to almost 50%, and at age 10/11 this is now over 70%. This development far exceeds the type of normal age-related changes we might expect from non-synesthetic children in the same task, as evidenced by the performance of our average- and high-memory controls (see **Figure 2**). Our study also allows us to confirm that the rate of acquiring fixed colors for letters slows down as synesthetic children age (see **Figure 2**). Synesthetes acquired on average 6.4 graphemes over the 12 months between Sessions 1 and 2, but only another 4.2 graphemes across the entire three years that followed. A more linear acquisition would have predicted that synesthetes would have acquired all 36 grapheme-color associations by age 10/11 years. Instead, we found that synesthetes are still below the 100% consistency that can characterize fully mature grapheme-color synesthesia, although our children are developing at different rates. Despite a the group-mean of ∼70%, synesthete MA already has 80% of his digits fixed with color, while GT has 92% of her letters. Hence although it may take several more years before these children achieve their full complement of grapheme-color associations, some are likely to achieve this sooner than others. It remains a point of investigation to track these children into stable adult-like profiles before we can appreciate the complete picture of the development of grapheme-color synesthesia.

This third installment of findings on the development of grapheme-color synesthesia provides valuable insights because it shows how synesthesia unfolds over time. Another recent study has also provided compatible data on development, this time in a far younger population. Spector and Maurer (unpublished, reported in Maurer et al., 2013) conducted a longitudinal study testing three pre-school children who were aged 3.5–4.5 years when the test began. These children had synesthetic mothers and so were reasonably likely to be synesthetes themselves, given familial inheritance patterns (e.g., Ward and Simner, 2005). Spector and Maurer provided 96 colored crayons, and children colored up to one grapheme per day and then immediately repeated the cycle, between three and six times. These three putative synesthetes were far more consistent on their choice of colors across cycles than age-matched controls (who were not at all consistent), and their consistency increased as the cycles advanced; children were 40% consistent in the first two cycles, aged 3.75–4.75, but were around 75% consistent in the final two cycles, age 4.5–5.5. This very high consistency score at age only 5.5 years, in the face of our own data, suggests these may be somewhat special synesthetes. Our own study here of randomly sampled child synesthetes predicts that consistency around age 5 should be less than 30%. We see two possible interpretations: either this study happened to sample particularly synesthetic children, or the task itself—with its repetitive coloring of graphemes performed on between 78 and 156 different days depending on the child may have reinforced colors earlier than otherwise expected. A final possible interpretation is that the children self-referred by research-interested synesthetic parents may be non-representative of more randomly sampled children *per se*. What is interesting, however, is that both this study and our own shows the development of synesthesia from more fluctuating to less fluctuating colors.

The issue of self-referral is an important one, on which we will therefore dwell a little longer. Other studies have also investigated childhood synesthesia, but again used samples of children likely to have been brought forward for testing by their synesthetic parents. Green and Goswami (2008) showed that a synesthetic population of children show cognitive benefits (e.g., in vocabulary and mathematics). Although Green and Goswami provided valuable information about the cognitive abilities of these children, it is not clear how much their findings can be extended to child synesthetes in general. It is likely that the children of parents with an interest in science, and the engagement in science, and in synesthesia in general, may have children who naturally score higher in the types of tests used in this study in any case. Hence, we are currently testing our own population, of randomly sampled synesthetes, on a range of test of cognitive tests in the hope this will reveal the true abilities of child synesthetes randomly sampled.

Although we have successfully identified a number of child synesthetes and tracked their development, we must acknowledge the shortfalls of our methodology based on the same reasons identified by Simner et al. (2009). In particular, we point out that our methods will necessarily underestimate the prevalence of childhood synesthesia, and this is because it relies on the test of consistency over time. Although this is a reliable test for adult synesthetes, who are generally highly consistent (e.g., 100%) in their synesthetic associations, our study itself shows that child synesthetes only acquire this consistency with time. Hence any study aiming to identify child synesthetes by their consistency is using a necessarily weak metric (see Simner et al., 2009 for full discussion). Furthermore, recent papers (Cohen Kadosh and Terhune, 2012; Eagleman, 2012; Simner, 2012) have asked whether high consistency is a true hallmark of synesthesia at all, or whether instead it characterizes only a subset of synesthetes. In other words, although all contemporary studies identify synesthetes by their consistency over time there may yet be some synesthetes overlooked by these studies who have varying and fluctuating colors, even into adulthood. Given both of these considerations, our study may have identified only a subset of child synesthetes with particularly early-developing synesthesia, or with particularly consistent synesthesia. Nonetheless, our study remains informative about this particular population *per se*.

Our study might be considered alongside theories about neurodevelopment in synesthesia. We have seen that grapheme-color synesthesia already exists in a nascent state at age 6 years, and that it develops slowly through childhood. Studies of adult graphemecolor synesthetes suggests it arises through "cross-activation" between adjacent brain regions involved, on the one hand, in the recognition of graphemes and words (the "visual word form area"; VWFA) and on the other, in color processing (e.g., V4; Hubbard et al., 2005; Rouw and Scholte, 2007). Importantly, since grapheme-color synesthesia relies on knowledge of culturally acquired symbols, it could not exist as such in in infancy. Hence, although the left-hemisphere language areas are already well-organized for the perception of spoken language in infants (Dehaene-Lambertz et al., 2006a,b) it is only later, when learning letters and numerals, that the types of representations required to drive grapheme-color synesthesia can be established. Here we see grapheme-color synesthesia developing in behavioral terms, alongside these children's growing familiarity with letters and digits.

In summary, we have identified child synesthetes, and observed their development longitudinally over three testing sessions spanning four years. We have seen that their synesthesia develops from relatively fluid pairings of graphemes to colors, which form more fixed associations at later ages. We have seen cases of synesthesia perhaps dying out over time, or developing more slowly in some individuals over others. We have been able to say with relative certainty that six children in our sample are synesthetes, and that a further six may be potential cases to be observed in future studies. Indeed, a number of questions raised here can be addressed by future testing, and we have one such study in preparation. Our participants were 10/11 years old at the time of testing for the current paper but are now 14/15 years of

#### **REFERENCES**


C. Robertson, and N. Sagiv (New York, NY: Oxford University Press), 11–33.


age. We therefore anticipate that their status as synesthetes might now be further informed by their own meta-awareness. Older synesthetes are able to self-report whether they have synesthetic experiences in a way that is far less tractable in younger children (and hence self-disclosure of synesthesia was not used as part of the methodology in the current study). Future observations of these synesthetes into adulthood might therefore rely on both objective and subjective reporting. Moreover, future studies might ask whether or to what extent similar developmental trajectories also characterize other forms of synesthesia, such as colors triggered by sound/music for example. Sound-color synesthesia relies on different "cultural artifacts," such as cultural dictates for musical meter and pitch structure, characteristics which are learned even in infancy (Soley and Hannon, 2010). Such differences in development with respect to inducers might therefore be expected to cause different developmental trajectories in the synesthesia itself. We hope that future studies might find ways to overcome existing obstacles in recruiting of child synesthetes, which have thus far hindered developmental research in the field. We also anticipate that future studies could provide information about the implications of synesthesia on other aspects of cognitive development. Green and Goswami (2008) suggest that young synesthetes might have a particular cognitive profile different from their peers. In addition, Simner et al. (2009) showed that synesthesia is relatively common in schools, with on average two to five grapheme-color synesthetes in every primary schools in the United Kingdom and United States respectively. If synesthesia can be linked to a particular cognitive profile (i.e., assets or deficits), as current adult studies *also* suggest (e.g., Meier and Rothen, 2013) then it will become more important to identify and understand synesthesia in childhood, and to address the particular educational needs of synesthetic children.


style. *Front. Psychol.* 4:632. doi: 10.3389/fpsyg.2013.00632


48–52. doi: 10.1016/j.bbr.2011. 03.071


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Received: 02 August 2013; accepted: 04 September 2013; published online: 12 November 2013.*

*Citation: Simner J and Bain AE (2013) A longitudinal study of grapheme-color synesthesia in childhood: 6/7 years to 10/11 years. Front. Hum. Neurosci. 7:603. doi: 10.3389/fnhum.2013.00603 This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2013 Simner and Bain. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## Developmental aspects of synaesthesia across the adult lifespan

## *Beat Meier 1,2 \*, Nicolas Rothen3 and StefanWalter 1,2*

*<sup>1</sup> Institute of Psychology, University of Bern, Bern, Switzerland*

*<sup>2</sup> Center for Cognition, Learning and Memory, University of Bern, Bern, Switzerland*

*<sup>3</sup> Department of Psychology, Sackler Centre for Consciousness Science, University of Sussex, Brighton, UK*

#### *Edited by:*

*Julia Simner, Univeristy of Edinburgh, UK*

#### *Reviewed by:*

*Gregor Thut, University of Glasgow, UK Julia Simner, Univeristy of Edinburgh, UK*

#### *\*Correspondence:*

*Beat Meier, Institute of Psychology, University of Bern, Fabrikstrasse 8, 3012 Bern, Switzerland e-mail: beat.meier@psy.unibe.ch*

In synaesthesia, stimuli such as sounds, words or letters trigger experiences of colors, shapes or tastes and the consistency of these experiences is a hallmark of this condition. In this study we investigate for the first time whether there are age-related changes in the consistency of synaesthetic experiences.We tested a sample of more than 400 graphemecolor synaesthetes who have color experiences when they see letters and/or digits with a well-established test of consistency. Our results showed a decline in the number of consistent grapheme-color associations across the adult lifespan. We also assessed age-related changes in the breadth of the color spectrum. The results showed that the appearance of primary colors (i.e., red, blue, and green) was mainly age-invariant. However, there was a decline in the occurrence of lurid colors while brown and achromatic tones occurred more often as concurrents in older age.These shifts in the color spectrum suggest that synaesthesia does not simply fade, but rather undergoes more comprehensive changes.We propose that these changes are the result of a combination of both age-related perceptual and memory processing shifts.

**Keywords: consistency, color perception, age-related changes, synaesthesia attrition**

## **INTRODUCTION**

Synaesthesia is a relative rare variation of human experience which involves the automatic activation of an unusual concurrent sensation in response to an inducing stimulus, for example a color experience in response to a letter printed in black. The condition runs in families, thus suggesting a genetic basis, and it emerges early in development (Asher et al., 2009; Simner et al., 2009). The particular associations (i.e., inducer-concurrent pairs such as grapheme-color) are typically idiosyncratic at the individual level and stable across time. In fact, the consistency of the synaesthetic associations has been proposed as a defining characteristic of synaesthesia (Baron-Cohen et al., 1987; Cytowic and Eagleman, 2002; Rich et al., 2005; Asher et al., 2006; Simner et al., 2006). Despite this, so far no study has investigated whether consistency changes across the adult lifespan and the goal of the present study was to fill this gap.

The question whether consistency may change across the lifespan is particularly relevant for those types of synaesthesia which involve color as a concurrent, because there is clear evidence for a decrease of chromatic sensitivity in older age (Knoblauch et al., 2001; Kinnear and Sahraie, 2002; Paramei, 2012). In contrast, as synaesthesia is not a purely perceptual phenomenon, it is possible that color-associations are habitually retrieved from memory. As automatic retrieval from memory seems to be age-invariant (Meier et al., 2013) the consistency of synaesthetic perception may not be prone to age-related decline. Moreover, there is evidence that late-blind synaesthetes maintain their synaesthetic visual percepts for years after blindness, thus, synaesthesia can even persist with little or no natural sensory

experience and independent from continuous associative learning (Steven and Blakemore, 2004).

In this study, we will first provide an outline that there is indeed evidence for plasticity of synaesthesia (i.e., the neonatal synaesthesia hypothesis, evidence for the development of grapheme-color synaesthesia in school age, reports of synaesthesia attrition in adolescence, variation and loss of synaesthetic experiences related to stress and brain damage). These findings motivate the present study in which we investigate changes in synaesthetic experiences over the adult lifespan, despite many studies emphasizing its stability in adult age.

According to the neonatal hypothesis of synaesthesia (Maurer and Maurer, 1988), we all may have been synaesthetes as young infants due to the increased functional connectivity in the infant brain. Synaesthesia in adults may than be viewed as a result of incomplete pruning or decreased inhibition of feedback projections in early development. While there is evidence for exuberant anatomical connectivity and for arbitrary sensory crossactivations in young infants (Huttenlocher et al.,1982;Wagner and Dobkins, 2011), and while there are early sensory cross-activations which may survive in the form of crossmodal correspondences (Spector and Maurer, 2011), the neonatal hypothesis of synaesthesia has been questioned recently (see Deroy and Spence, 2013, for a critical discussion).

In order to investigate the development of grapheme-color synaesthesia in real time within a childhood population over an extended period, Simner et al. (2009)sampled more than 600 children between 6 and 7 years. To assess consistency, the children were presented with the letters of the alphabet and the digits 0–9, one by one, in a random order on a computer screen together with an on-screen palette of 13 colors. They were required to pair each grapheme with the "best" color. After the presentation of all the graphemes, there was a short break and then the same test was repeated again. By comparing the choices of the two tests a consistency score was calculated for each child as the number of identical grapheme-color choices. On average this consistency score was 3.5 out of 36 graphemes. Next, 47 children who had scores significantly higher than this mean score were identified as potential synaesthetes and were retested in a second session 1 year later (i.e., at the age of 7/8 years). Using the same test procedure, Simner et al. (2009) identified eight children who scored highly consistent both within and across the two test sessions. These were considered as genuine synaesthetes. The other 39 children were considered as high-memory non-synaesthetes. Moreover, Simner et al. (2009) found that the number of graphemes that triggered consistent synaesthetic experiences – the bandwidth of synaesthesia (Asher et al., 2006) – increased with age. On average, the genuine synaesthetes had acquired 6.4 new grapheme-color associations over the 1-year period (i.e., they had a mean consistency score of 10.5 in Session 1 and of 16.9 in Session 2).

In a follow-up study, Simner and Bain (2013) tested these children again at age 10/11 with the same procedure as before in order to establish whether the synaesthetes performed still consistently after another 3 years – or whether synaesthesia would "die out" in some individuals which may be consistent with the neonatal hypothesis – and whether the number of consistent grapheme-color associations would further increase. Most important, the results showed that five of the eight synaesthetes identified in Simner et al. (2009) still conformed to the synaesthesia criteria. Moreover, the number of consistent graphemes for the genuine synaesthetes increased again, to 25.7 out of 36. Overall, the results of the two studies suggest that grapheme-color synaesthesia can be assesses already in 6-year old children and that the number of consistent inducerconcurrent pairs increases with development. The latter result shows that the bandwidth of the synaesthesia also increases during development (cf., Asher et al., 2006). Moreover, the results also suggest that synaesthesia can disappear during childhood, which would be consistent with the neonatal synaesthesia hypothesis.

Besides this empirical indication for the disappearance of synaesthesia in childhood, there is also evidence for synaesthesia attrition later in development. For example, there are several anecdotal reports of cases that seem to have had synaesthesia as children, but who have lost them during adolescence (Flournoy, 1893; Riggs and Karwoski, 1934; Cytowic, 1997; Emrich et al., 2000). In line with these reports, one of the authors of the present article was approached by a 25-year old student, who claimed to have had synaesthetic grapheme-color experiences as a child but had lost them during adolescence. Interestingly, she reported that she still remembered the color associations with high certainty. When tested with a synaesthetic Stroop test, she showed a reliable interference effect. The synaesthetic Stroop test involves the presentation of colored graphemes that are either congruent or incongruent to the grapheme-color association of

a particular synaesthete and the participant is required to name the color of the grapheme as quickly as possible. This test has often been used to demonstrate the genuineness of synaesthesia, because synaesthetes show slower responses to incongruent compared to congruent colors, while non-synaesthetes do not show this effect (e.g., Mills et al., 1999; Odgaard et al., 1999; Dixon et al., 2004; Ward et al., 2007). However, it has been demonstrated that this effect can be induced through training grapheme-color associations in non-synaesthetes (Elias et al., 2003; Meier and Rothen, 2009; Rothen et al., 2013a; Rothen and Meier, 2014).

Further evidence for the variability of synaesthetic experiences comes from studies with adults. Rich et al. (2005) noted that "although some synaesthetes reported that the frequency or intensity of their synaesthetic experiences had diminished with age, most reported no change in their synaesthesia since childhood" (p. 67). Moreover, 28% reported that stress, alcohol, and other drugs influenced synaesthesia (either attenuating or enhancing the experience). Consistently, in a large sample of graphemecolor synaesthetes, Eagleman found similar results (in Cytowic and Eagleman, 2002, p. 139).

In an autobiographical review, Day (2013)reported that he had lost his synaesthetic experiences due to post-traumatic stress disorder (PTSD), that is, after getting into an earthquake while being trapped in the 17th floor of a shaking building with cracks running through the ceiling and the walls. He reported that although he realized this loss immediately, due to PTSD it did not start to concern him until about 2 months later. In order to revive the synaesthesia, he tried to "produce some by listening to music" (p. 918). After about 3 months, he started to worry about whether the synaesthetic experiences would come back at all, and soon later the experiences began to return very slowly, starting off very faintly and "all washed out." It took him at least another 3 months until the synaesthetic experiences were back as before the traumatic event.

In Dittmar (2007), a woman reported the loss of her synaesthesia due to a seizure. After the incidence, she was half-sided paralyzed and also suffered from disorientation due to the changed world of experiences without synaesthesia. For her, this was particularly disturbing because she had used her synaesthesia to support her memory and planning. With rehabilitation, she was able to regain her cognitive capacity, concentration, and endurance. Concurrently, her synaesthesia came back, but not exactly as before. For example, some of her grapheme-color associations had shifted. Moreover, she reported that the intensity of the synaesthetic experience was more variable and that the experiences faded when she was under stress.

Spalding and Zangwill (1950) reported the case of a 24 year old man who was shot in the head. Besides having a large deficit in his visual memory, he reported the loss of his sequenced order of numerals and, as a consequence, innumeracy. Spalding and Zangwill supposed that the patient had lost his sequence-space synaesthesia and assumed that this was the primary cause for his disturbance.

Sacks (1997) reported the case of a painter who lost his ability to see colors after a car accident. This also resulted in a loss of his sound-color synaesthesia and he reported that listening to music had become flat and atrophied. Thus, synaesthetic experiences are variable in adult age and can even disappear forever after a brain injury.

Unfortunately, these reports did not provide detailed information about the particular brain areas that were damaged and caused the loss of synaesthesia. However, there are modern and safe techniques in neuroscience that allow the temporal suppression of specific brain areas with transcranial magnetic stimulation (TMS). For example, Esterman et al. (2006) used repetitive TMS over the posterior parietal lobes and showed that the synaesthetic Stroop effect was eliminated when applied over the right hemisphere. In contrast, TMS to the visual area V1 did not affect the synaesthetic Stroop effect. Similarly, Muggleton et al. (2007) found that single-pulse TMS over the right parieto-occipital cortex disrupted synaesthetic Stroop interference and there was also a marginally significant effect for the left parieto-occipital site, while TMS over other parietal areas showed only minimal performance disruption. Moreover, Rothen et al. (2010) showed that TMS over the parietal-occipital cortex is also effective at suppressing implicit bidirectional effects of synaesthesia.

In a recent study, we investigated whether the application of rTMS over right parietal cortex would eliminate synaesthetic experiences (Meier et al., in preparation). This question was motivated by a grapheme-color synaesthete who had explicitly expressed the wish to experience the world temporally without her synaesthesia. In order to ensure that the TMS protocol affected synaesthesia as planned we also administered the synaesthetic Stroop task before and after TMS application. We were also interested whether TMS would affect the specific grapheme-color associations and for this purpose, we administered a consistency test that required grapheme-color matches on a continuous color palette 1 week before, immediately after the TMS application and 1 week later (cf., Meier and Rothen, 2013). As expected, the results showed a synaesthetic Stroop effect before but not immediately after TMS application. However, to a big disappointment of the synaesthete, she did not experience any change in her own experience. Although she did not experience a change of her synaesthetic experiences, the consistency of the specific color experiences was reduced. That is, consistency was higher across the 2 weeks interval (i.e., 1 week before and 1 week after the TMS application) than both scores with a 1 week interval (i.e., involving the TMS application). Thus, in the latter situation, TMS seems to have influenced the synaesthetic color experiences such that she had to retrieve the specific colors from memory. Overall, the results of the TMS studies suggest that grapheme-color synaesthesia relies specifically on parietooccipital pathways similar to normal color perception and that these regions are not only involved in explicit but also in implicit synaesthetic binding (Ramachandran and Hubbard, 2001; Hubbard and Ramachandran, 2005; Esterman et al., 2006; Rothen et al., 2010).

To summarize, there is evidence from multiple sources that the synaesthetic experiences can vary during adulthood and there are even conditions under which synaesthesia can disappear completely, either transiently or forever. There is also converging evidence from TMS-studies, as well as from functional and structural imaging studies that the occipito-parietal lobes are critically involved in grapheme-color synaesthesia (Hubbard et al., 2011; Rouw et al., 2011; Specht, 2012). To our knowledge no study has yet addressed the developmental trajectory of synaesthesia in adult age. As many cognitive functions are subject to age-related changes, it is possible that synaesthetic experiences and their consistency decline with age. However, although the occipito-parietal lobes are subject to age-related decline, this is much smaller than the decline in frontal or temporal brain areas (e.g., Brickman et al., 2006). Moreover, there is also evidence that synaesthesia can spread to novel inducers and thus it is possible that over the lifetime these opportunities accumulate (Rich et al., 2005; Mroczko et al., 2009; Blair and Berryhill, 2013; Brang et al., 2013). If so, it is possible that the bandwidth of synaesthesia may even increase across the adult lifespan. These questions are addressed in the present study. Specifically, using the method introduced by Simner et al. (2006, 2009), Rothen and Meier (2010), and Simner and Bain (2013), we investigated in a sample of more than 400 grapheme-color synaesthetes whether the number of consistent grapheme-color associations changes across the adult lifespan.

## **MATERIALS AND METHODS**

#### **PARTICIPANTS**

The sample was recruited from our Synaesthesia-Check database. The Synaesthesia-Check is a short questionnaire used to establish contact with the general public interested in our research (www.synaesthesie.unibe.ch). It involves questions about potential forms of synaesthesia, the nature of synaesthetic experiences, and it provides the opportunity to leave contact information for those willing to take part in future studies. From those who had left an email address, we contacted a total of 1233 persons, who had indicated in the Synaesthesia-Check that they had consistent and involuntary synaesthetic experiences. From these, 631 persons responded to our invitation to participate in the present study. As we will focus on grapheme-color synaesthesia, we report the data of those 439 participants who indicated they had grapheme-color synaesthesia, that is, either color experiences in response to digits only (*N* = 17), letters only (*N* = 48) or both digits and letters (*N* = 374). The mean age was 38.03 years (*SD* = 13.8, range 18–91), 89.7% of them were females, and 88.4% of them were right-handed.

#### **PROCEDURE AND MATERIALS**

Participants were invited to click on a link in an email message to participate in this study which was announced as an investigation of their synaesthetic experiences. After giving consent, they were asked which forms of synaesthesia they had. Specifically, they had to indicate, separately, whether they had color experiences in response to letters, digits, words, music, and sounds. Those who had indicated color experiences to letters and digits were forwarded to a grapheme-color consistency test. The method for the consistency test was adopted from Simner et al. (2006, 2009), Rothen and Meier (2010), and Simner and Bain (2013). A computerized test individually presented 36 graphemes (A–Z; 0–9), in black on a white background, in a random order, with each presentation accompanied by a palette of 13 colors (black, dark blue, brown, dark green, gray, pink, purple, orange, red, white, light blue, light green, and yellow) and a "no color" option. The arrangement of the colors within the palette was randomized on every trial, and participants were required to select the matching color for each grapheme. An example trial is presented in **Figure 1**.

Participants were instructed to choose the color that fits best to the specific letter or digit. They were informed that if no color of the palette exactly matched their synaesthetic experience they should select the color that came closest. In the case that they had no synaesthetic color experience for a particular grapheme, they could use the "no color" option. After the presentation of all the 36 letters and digits, an immediate retest was administered in which the order of the graphemes was re-randomized.

#### **ANALYSIS**

Digits and letters were analysed separately. As the graphemes were presented in black color on the screen, we did not consider "black" responses as synaesthetic experiences. First, the number of consistent color choices in the test and re-test were counted and a correlational analysis was performed to assess the potential relationship between age and the number of consistent colors. Next, we split the sample into three different age groups and we used analyses of variance to gain more fine-grained insights into the trajectory of the consistency scores.

In a second set of analyses, we investigated changes in the breadth of the color spectrum, separately for digits and letters. Specifically, we investigated age-related changes in the frequency in which each of the 12 colors occurred as a concurrent in each of the three age groups.

#### **RESULTS**

#### **CONSISTENCY OF COLOR EXPERIENCES**

Scatterplots of the relationship between age and the number of consistent color experiences (consistency score, CS) are presented in **Figure 2**, separately for digits (**Figure 2A**) and for letters (**Figure 2B**). For statistical analyses alpha was set to 0.05.

Correlational analyses revealed a significant negative relationship between age and the number of consistent color associations with *r* = −0.15, *p* < 0.01 for digits and *r* = −0.18, *p* < 0.01 for letters. Thus, for both digits and color there was a decrease in the number of consistent synaesthetic experiences across the lifespan.

In order to further investigate this decrease in consistency, we created three age groups (i.e., younger group aged 18–28 years, a middle group aged 29–42, and older group aged 43–91) with approximately equal sample sizes (i.e., *N* = 135, 135, and 152 for the digit-color associations and *N* = 132, 122, and 137 for lettercolor associations). We considered this as a conservative approach that should eliminate spurious results due to outliers, particularly in older age.

The mean CS across the three age groups are presented in **Figure 3**, separately for digits and letters. The results of two separate one-way analyses of variance (ANOVA) showed significant age effects for both digits and letters, with *F*(2,421) = 5.80, *p* < 0.01, and *F*(2,390) = 4.26, *p* < 0.05. For digits, *post hoc* tests revealed that the consistency scores of the younger adults were higher than both the middle and older adult groups (*p*s < 0.01), while the latter two groups were not statistically different (*p* = 0.64). Similarly, for letters, *post hoc* tests revealed that the age effect was due to the higher consistency scores of the younger adults compared to the older adults (*p* < 0.01). No other effect was significant (*p*s = 0.13 and 0.18, respectively).

Next, we analysed the number of graphemes without colors (GWC), that is, those for which participants consistently chose "no color" or consistently chose the "black" color option. The GWC was *M* = 0.81 for digits (0.63, 0.98, and 1.01 for young, middle and older adults) and *M* = 3.81 for letters (2.72, 3.99, and 4.72). ANOVAs revealed an age-effect, *F*(2,421) = 6.421, and *F*(2,390) = 5.52, both *p*s < 0.01, for digits and letters, respectively. Compared to younger adults, the number of GWC was higher in older participants. For digits, *post hoc* tests showed that the young group differed from both the middle and older groups (*p*s < 0.01), while the latter did not. Similarly, for letters, *post hoc* tests showed that the young group differed from both the middle and older groups (*p*s < 0.05), and no other effect.

These findings suggest that rather than consistency *per se*, it is the bandwidth of synaesthesia that declines across the adult lifespan. Synaesthetic attrition is expressed as a reduction in the number of graphemes which trigger color experiences. To further test this interpretation, we calculated a corrected consistency score (CCS) for each participant that takes into account the number of GWC: CCS = CS/(number of possible graphemes – GWC). The number of possible graphemes is 10 for digits and 26 for letters. If our interpretation holds, then no age-effects would be expected for the CCS. The results showed that for digits, the CCS were 87.1, 82.9, and 84.3 for the young, middle, and older group. For letters, these were 78.5, 78.6, and 76.9% for the young, middle, and older group. ANOVAs failed to reach significance, *F*(2,421) = 1.83, *p* = 0.17, and *F*(2,390) < 1, *p* = 0.70, for digits and letters, respectively. Thus, these results corroborate our hypothesis that synaesthetic attrition is mainly driven by a decrease in the number of graphemes that trigger a synaesthetic color experience. If a particular grapheme

triggers a synaesthetic experience, this experience is still consistent, suggesting that attrition is not due to a reduction in consistency but rather due to reduced bandwidth, thus mirroring the developmental trajectory observed in childhood (Simner and Bain, 2013).

Overall, the results from the correlational analyses and from the age-group comparisons suggest that there is a decrease in the bandwidth of synaesthetic experiences across the adult lifespan. We hypothesized that, if this interpretation is true, it is highly likely that this decrease is also reflected in the pattern of colors that constitute grapheme-color synaesthesia.

#### **FREQUENCY OF CONSISTENT COLORS**

In a next set of analyses, we analysed how often a particular color had been chosen as a concurrent. Although this analysis was mainly exploratory, based on anecdotal reports of decreased intensity of synaesthetic color experiences and on the empirical results of age-related decline in color perception and color discrimination (Fiorentini et al., 1996; Paramei, 2012), we expected that lurid colors such as yellow, pink, or orange might be mostly affected.

In order to test whether the age-related trajectory of the 12 colors differed at all, we calculated a two-factorial ANOVA with age cohort (young, middle, older) and color (red, blue, green, yellow, orange, violet, light blue, light green, magenta, brown, silver-gray, white), separately for digits and for letters. The interaction was highly significant with *F*(22,4609) = 3.638, *p* < 0.001, for digits and with *F*(22,4268) = 3.627, *p* < 0.001, for letters, respectively, which demonstrates that the age-related trajectory differed across colors.

The mean number of consistent color associations for digits are presented in **Figure 4**. An ANOVA of the number of consistent *red color* experiences suggested a significant group difference, *F*(2,421) = 4.95, *p* = 0.008, and *post hoc t*-tests indicated that the consistency scores of the younger adults were higher than the scores of the middle adult group, *p* = 0.002; no other effect

reached significance. The same kind of ANOVA for the number of consistent *blue color* experiences suggested a significant group difference, *F*(2,421) = 3.41, *p* = 0.034, and *post hoc* tests indicated that the consistency scores of the middle adults were lower than the scores of the other two groups, *p* = 0.014 compared to the young, and *p* = 0.044 compared to the older group, but no other effect

reached significance, that is, no age-specific decline. The number of consistent *green color* experiences did not differ across groups, *F*(2,421) = 0.31, *p* = 0.73.

AnANOVA of the number of consistent *yellow color* experiences suggested an age-effect *F*(2,421) = 5.12, *p* = 0.006, and *post hoc* tests indicated that the consistency scores of the young group were higher than the scores of the middle and older group, *p* = 0.012 and 0.003, respectively, but no other difference. For *orange color* experiences, the ANOVA also suggested a group difference, *F*(2,421) = 4.96, *p* = 0.007, and *post hoc* tests indicated that the consistency scores of the older adults were lower than those of the two younger groups, both *p*s = 0.001, while the latter groups did not differ, *p* = 0.93. For *violet color* experiences, the ANOVA also suggested an age-related effect, *F*(2,421) = 7.73, *p* = 0.001, with *post hoc* tests indicating that the consistency scores of the young adults were higher than those of the other two groups, with *p* = 0.009 and 0.005, respectively, while the latter groups did not differ, *p* = 0.88.

Neither for *light-blue* nor for *light-green* did the ANOVA suggest a group difference, with *F*(2,421) = 0.47, *p* = 0.62, and *F*(2,421) = 0.26, *p* = 0.77. In contrast, for *magenta* the ANOVA suggested a group difference, *F*(2,421) = 3.61, *p* = 0.03, and *post hoc* tests indicated that the consistency scores of the young adults were higher than those of the older groups, *p* = 0.008, but no other difference.

In contrast, theANOVA of the number of consistent *brown color* experiences suggested an age-related increase, *F*(2,421) = 6.94, *p* = 0.001, and *post hoc* tests indicated that the older group scored higher than the young and the middle group, *p* < 0.001 and 0.031, respectively, but no other difference. For *silver-gray color* experiences, the same pattern emerged, *F*(2,421) = 5.98, *p* = 0.003, and again *post hoc* tests indicated that the consistency scores of the older adults were higher than those of the younger group, *p* < 0.001, but no other difference. For *white color* experiences, the ANOVA suggested no group difference, *F*(2,421) = 0.28, *p* = 0.76.

We performed the same kind of analyses for letters. The mean number of consistent color associations for letters are presented in **Figure 5**. ANOVAs of the number of consistent *red color* experiences, *blue color* experiences, and *green color* experiences did not suggest any group differences, *F*(2,390) = 0.44, *p* = 0.64, *F*(2,390) = 0.05, *p* = 0.95, *F*(2,390) = 0.64, *p* = 0.53, respectively.

The ANOVA of the number of consistent *yellow color* experiences suggested an age-effect *F*(2,390) = 13.85, *p* < 0.001, and *post hoc* tests indicated that the young group differed from the middle and older group, *p* = 0.001 and *p* < 0.001, respectively, while the latter two groups did not, *p* = 0.077. For *orange color* experiences, the ANOVA did not suggest a group difference, *F*(2,390) = 1.63, *p* = 0.197. However, for *violet color* experiences, the ANOVA suggested a group difference, *F*(2,390) = 6.81, *p* = 0.001, and *post hoc* tests indicated that the consistency scores of the young adults were higher than those of the other two groups, with *p* = 0.03 and *p* < 0.001, respectively, while the latter groups did not differ, *p* = 0.16.

For *light-blue*, the ANOVA suggested a group difference, *F*(2,390) = 4.51, *p* = 0.012, and *post hoc* tests denoted that the consistency scores of the young adults were higher than those of

the older group, *p* = 0.003, but no other effect. For *light-green,* the ANOVA also suggested a group difference, with *F*(2,390) = 3.72, *p* = 0.025, and *post hoc* tests indicated that the consistency scores of the young adults were higher than those of the other two groups, with *p* = 0.05 and 0.009, respectively, while the latter groups did not differ, *p* = 0.56. For *magenta,* the ANOVA also suggested a group difference, *F*(2,390) = 4.16, *p* = 0.016, and *post hoc* tests denoted that the consistency scores of the young adults were higher than those of the middle and older groups, with *p* = 0.03 and 0.007, respectively, while the latter groups did not differ, *p* = 0.64.

Neither for *light-blue* nor for *light-green* did the ANOVA suggest a group difference, with *F*(2,421) = 0.47, *p* = 0.62, and *F*(2,421) = 0.26, *p* = 0.77. In contrast, for *magenta* the ANOVA suggested a group difference, *F*(2,421) = 3.61, *p* = 0.03, and *post hoc* tests indicated that the consistency scores of the young adults were higher than those of the older groups, *p* = 0.008, but no other difference.

In contrast, theANOVA of the number of consistent *brown color* experiences suggested an age-related increase, *F*(2,421) = 6.94, *p* = 0.001, and *post hoc* tests indicated that the older group scored higher than the young and the middle group, *p* < 0.001, and *p* = 0.031, respectively, but no other difference. For *silver-gray color* experiences, the same pattern emerged, *F*(2,421) = 5.98, *p* = 0.003, and again *post hoc* tests indicated that the consistency scores of the older adults were higher than those of the younger group, *p* < 0.001, but no other difference. For *white color* experiences, the ANOVA suggested no group difference, *F*(2,421) = 0.28, *p* = 0.76.

We performed the same kind of analyses for letters. The mean number of consistent color associations for letters are presented in **Figure 5**. ANOVAs of the number of consistent *red color* experiences, *blue color* experiences, and *green color* experiences did not suggest any group differences, *F*(2,390) = 0.44, *p* = 0.64, *F*(2,390) = 0.05, *p* = 0.95, *F*(2,390) = 0.64, *p* = 0.53, respectively.

The ANOVA of the number of consistent *yellow color* experiences suggested an age-effect *F*(2,390) = 13.85, *p* < 0.001, and *post hoc* tests indicated that the young group differed from the middle and older group, *p* = 0.001, and *p* < 0.001, respectively, while the latter two groups did not, *p* = 0.077. For *orange color* experiences, the ANOVA did not suggest a group difference, *F*(2,390) = 1.63, *p* = 0.197. However, for *violet color* experiences, the ANOVA suggested a group difference, *F*(2,390) = 6.81, *p* = 0.001, and *post hoc* tests indicated that the consistency scores of the young adults were higher than those of the other two groups, with *p* = 0.03 and *p* < 0.001, respectively, while the latter groups did not differ, *p* = 0.16.

For *light-blue*, the ANOVA suggested a group difference, *F*(2,390) = 4.51, *p* = 0.012, and *post hoc* tests denoted that the consistency scores of the young adults were higher than those of the older group, *p* = 0.003, but no other effect. For *light-green,* the ANOVA also suggested a group difference, with *F*(2,390) = 3.72, *p* = 0.025, and *post hoc* tests indicated that the consistency scores of the young adults were higher than those of the other two groups, with *p* = 0.05 and 0.009, respectively, while the latter groups did not differ, *p* = 0.56. For *magenta,* the ANOVA also suggested a group difference, *F*(2,390) = 4.16, *p* = 0.016, and *post hoc* tests

**FIGURE 5 | Number of consistent colors for letters across the adult lifespan, separately for each color (red, blue, green, yellow, orange, violet, light-blue, light-green, magenta, brown, gray, and white, from top left to bottom right).** \**p* < 0.05, # = number.

denoted that the consistency scores of the young adults were higher than those of the middle and older groups, with *p*=0.03 and 0.007, respectively, while the latter groups did not differ, *p* = 0.64.

The ANOVA of the number of consistent *brown color* experiences suggested no age-related effect, *F*(2,390) = 0.32, *p* = 0.73. Similarly, the ANOVA of the number of consistent *silver-gray color* experiences suggested no group difference, *F*(2,390) = 1.15, *p* = 0.32. In contrast, for *white color* experiences, the ANOVA suggested an age-related increase, *F*(2,390) = 4.979, *p* = 0.007, and *post hoc* tests denoted that the consistency scores of the older adults were higher than those of the young group, *p* = 0.002, but no other effect (*p*s > 0.11).

In order to summarize the results of the age-related trajectory for individual colors, we would like to order the findings according to three distinct patterns. First, several colors seem to show an age-related decrease, such as yellow, orange, violet, light-blue or magenta. Second, there are colors that do not seem to show consistent signs of age-related changes such as red, blue or green. Finally, there seem to be several colors that rather show an increase than a decrease such as brown and white (and eventually gray, at least for digits). This pattern of results would suggest that the most frequent color terms such red, green or blue are not subject to age-related changes. In contrast, less frequent color terms such as yellow, orange, violet or magenta, which may also be considered as more intensive, seem to occur less often as synaesthetic concurrents with increasing age. Finally, white, gray, and brown, typically considered as less jaunty (or achromatic) rather seem to increase across adult age.

#### **DISCUSSION**

This is the first study that investigated the developmental trajectory of the consistency of synaesthetic grapheme-color experiences across the adult lifespan. Specifically, using the method introduced by Simner et al. (2006), we tested a sample of more than 400 grapheme-color synaesthetes by asking them to choose the matching color for each digit and each letter of the alphabet in two separate test runs. We measured the number of consistent grapheme-color associations and assessed the relationship with age using both a correlational and a quasi-experimental approach. Our results showed a small but consistent age-related decline in the number of consistent grapheme-color associations. Together with previous results on the development of grapheme-color synaesthesia in children and adolescents in which an increase in the number of consistent grapheme color associations was found (Simner et al., 2009; Simner and Bain, 2013), the results of the present study suggest that the bandwidth of grapheme-color experiences is subject to a similar inverted u-shaped curve as many other cognitive functions (Craik and Bialystok, 2006; Zimmermann and Meier, 2006, 2010; Shing and Lindenberger, 2011; Weiermann and Meier, 2012; Meier et al., 2013).

We also assessed age-related changes in the breadth of the color spectrum. The results showed that the appearance of the most frequent color terms (i.e., red, blue, and green) was mainly age-invariant. However, lurid colors such as yellow, orange, and magenta occurred less often as synaesthetic concurrents with increasing age. In contrast, disimpassionated colors such as brown, gray, and white were chosen more often. These results underline

that the age-related decline in the consistency of grapheme-color associations is not simply a chance result. They seem to be connected with systematic age-related changes in color perception and discrimination (Fiorentini et al., 1996; Knoblauch et al., 2001; Kinnear and Sahraie, 2002; Paramei, 2012).

However, the question remains why the probability declines that certain inducers trigger synaesthetic concurrents with older age. One possibility is that the graphemes do not trigger the synaesthetic experiences anymore because color perception underlies an age-related decline. However, even so, due to the lifelong associations between the inducers and the concurrent, one would expect that there are strong semantic links such that synaesthetes still would "know" the colors of the graphemes. A perceptual vs. memory distinction has been made for projector vs. associator synaesthetes (Rouw and Scholte, 2010). Accordingly, one would predict a different age-related trajectory for associate vs. projector synaesthetes. As we have not included a projector vs. associator questionnaire in this study, we cannot test this hypothesis. However, this may be an interesting avenue for future research.

Another possibility is that synaesthetic associations change across time and thus, the number of consistent grapheme color associations decreases. While we cannot exclude this possibility, this explanation cannot account for the present results because the retest was within the same test session and thus a change in associations across this short time-window is not realistic. A further explanation is that with increasing age multiple concurrents develop for a particular inducer and thus, the synaesthetic associations are systematic but not stable. For example, Simner (2012) reported that a minority of synaesthetes fulfill most of the criteria for the condition, but fail to pass the test of consistency due to changing grapheme-color associations. Although this explanation is possible, it is still not clear why multiple concurrents would be more likely in older age. Moreover, as our results suggest that the number of inducers decreases with age, this explanation cannot fully account for the whole pattern of results.

Thus, an explanation that posits that with increasing age some graphemes do not trigger the synaesthetic colors anymore *and* the specific associations are also lost is more likely. Such an explanation would imply that lower-level perceptual processes as well as higher-level cognitive processes are involved in the change of synaesthetic consistency. Specifically, it is likely that with age the processing of lurid colors is reduced such that their sensation becomes weaker and eventually fades away. Moreover, it seems also that the association between an inducer and a concurrent may change, most likely toward pastel colors, and then eventually the association is forgotten over time. For example, a yellow concurrent may fade toward white, and later on, the concurrent white experience may be completely lost. There is wide-spread evidence for an age-related decline in memory processes, and under the assumption that some of the grapheme-color associations are re-established due to fading perceptual processes, it may be that these re-established associations are also most likely to be forgotten, similar to re-consolidated memories (Nader and Hardt, 2009). However, in order to provide stronger evidence for this explanation, longitudinal data are necessary.

We acknowledge that an important restriction of the present study is its cross-sectional nature. Cross-sectional studies are not suited to investigate individual trajectories and thus rather provide a snapshot of evidence. Moreover, a further drawback of crosssectional studies is their susceptibility to cohort effects. It may also be argued that the grapheme-color matching task draws on memory for the previously chosen association and as a consequence memory effects (potentially favoring the younger participants) may have influenced the age-related decline in the consistency measure. However, for the present study, we have no reason to assume any confounding cohort effects. Rather, we believe that by using an internet-based approach we may have biased our sample toward more active and highly motivated older adults and as such neither the decline in bandwidth nor the changes in the synaesthetic color experiences can be accounted for. Similarly, if the age-related decline in the consistency measure would be simply caused by memory effects, it would be reasonable to expect that this occurs uniformly across the color spectrum. Nevertheless, we believe that studies with a longitudinal design will lead to more detailed insights about the trajectory of synaesthesia attrition in future studies.

The main focus of this study was on the number of consistent grapheme-color associations. In line with previous studies we have used a pre-defined color palette which may not have represented the specific color experience for a particular grapheme for each synaesthete. In fact, several participants complained that the color palette was too small to capture their synaesthetic experiences. Despite this, many synaesthetes did not report synaesthetic color experiences for each and every grapheme. In fact, the age-related decline in the number of consistent grapheme-color experiences was mainly due to the fact that with increasing age fewer graphemes triggered synaesthesia. However, we would like to emphasize that the method used in the present study is not particularly suited to measure the specificity of the synaesthetic color experiences as they are used in other methods in which colors can be chosen from a much larger palette (e.g., Eagleman et al., 2007; Rothen et al., 2013b). It is an interesting direction for future research to address whether the specificity of the color experiences also changes with age. Similarly, future studies may also address the question of the intensity of the color experience more directly. So far, there is only limited evidence that the experience of color intensity changes, however, from the results of present study, we would also predict a decline in the experience of color intensity.

Rather than the specificity of synaesthetic associations, we have found that the bandwidth of grapheme-color synaesthesia was reduced across the adult lifespan. Although the bandwidth of synaesthesia is an important issue, so far, it has not received much attention. In a study on the revised test of genuineness, Asher et al. (2006) noted that broad-band synaesthetes reported percepts to a wide range of stimuli (80–100%) while narrowband synaesthetes report percepts to a smaller range of stimuli. There are also differences in bandwidth across different types of synaesthesia. For example, in sound-color synaesthesia there is an infinite amount of inducers, while for grapheme-color synaesthesia the maximum number is clearly defined (i.e., 26 for the roman alphabet and 10 for digits). For other forms of synaesthesia

the amount of possible inducers is much smaller, for example four for swimming-style synaesthesia (Nikolic et al., 2011; Rothen et al., 2013a) and only one for mirror-touch synaesthesia (see Rothen and Meier (2013)for a discussion whether the latter should be regarded as a form of synaesthesia at all). There have been suggestions that in the most common forms of synaesthesia, the bandwidth in adult synaesthetes is typically 80–100% (e.g., Ward et al., 2005; Simner and Bain, 2013). However, so far there is no clear criterion how many consistent synaesthetic grapheme-color experiences a person must have to be considered as a genuine synaesthete.

To summarize, this is the first study that has tested the synaesthesia attrition hypothesis in a large sample of grapheme-color synaesthetes. Our results showed a small but consistent decline in the bandwidth of synaesthesia across the adult lifespan and thus support the idea of synaesthesia attrition in older age. This result fits with an age-related decline in many other perceptual and cognitive functions. It also fits with evidence of an agerelated decline in other forms of cross-modality (Simner and Ludwig, 2012; Ludwig and Simner, 2013). Moreover, together with recent findings from developmental studies, our results suggest that the bandwidth of synaesthesia follows an inverted u-shape function (Simner et al., 2009; Simner and Bain, 2013). We also found age-related changes in the breadth of the color spectrum of the synaesthetic concurrents. Specifically, there was a decline in the occurrence of lurid colors while brown and achromatic tones occurred more often as concurrents in older age. These shifts in the color spectrum suggest that synaesthesia does not simply fade, but rather undergoes more comprehensive changes.

## **REFERENCES**


Dittmar, A. (2007). *Synästhesien: Roter Faden durchs Leben?* Essen: Die Blaue Eule.


Flournoy, T. (1893). *Des Phénomènes de Synopsie*. Paris: Alcan.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

#### *Received: 03 December 2013; accepted: 20 February 2014; published online: 11 March 2014.*

*Citation: Meier B, Rothen N andWalter S (2014) Developmental aspects of synaesthesia across the adult lifespan. Front. Hum. Neurosci. 8:129. doi: 10.3389/fnhum.2014.00129 This article was submitted to the journal Frontiers in Human Neuroscience.*

*Copyright © 2014 Meier, Rothen and Walter. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## ADVANTAGES OF PUBLISHING IN FRONTIERS

FAST PUBLICATION Average 90 days from submission to publication

COLLABORATIVE PEER-REVIEW

Designed to be rigorous – yet also collaborative, fair and constructive

RESEARCH NETWORK Our network increases readership for your article

## OPEN ACCESS

Articles are free to read, for greatest visibility

#### TRANSPARENT

Editors and reviewers acknowledged by name on published articles

GLOBAL SPREAD Six million monthly page views worldwide

#### COPYRIGHT TO AUTHORS

No limit to article distribution and re-use

IMPACT METRICS Advanced metrics track your article's impact

SUPPORT By our Swiss-based editorial team

EPFL Innovation Park · Building I · 1015 Lausanne · Switzerland T +41 21 510 17 00 · info@frontiersin.org · frontiersin.org