This article is part of the Research Topic Synaesthesia

Original Research ARTICLE

Front. Psychol., 17 June 2013 | doi: 10.3389/fpsyg.2013.00330

How well do you see what you hear? The acuity of visual-to-auditory sensory substitution

Alastair Haigh1, David J. Brown1, Peter Meijer2 and Michael J. Proulx3*
  • 1Crossmodal Cognition Laboratory, School of Biological and Chemical Sciences, Queen Mary University of London, London, UK
  • 2Metamodal BV, Eindhoven, Netherlands
  • 3Crossmodal Cognition Laboratory, Department of Psychology, University of Bath, Bath, UK

Sensory substitution devices (SSDs) aim to compensate for the loss of a sensory modality, typically vision, by converting information from the lost modality into stimuli in a remaining modality. “The vOICe” is a visual-to-auditory SSD which encodes images taken by a camera worn by the user into “soundscapes” such that experienced users can extract information about their surroundings. Here we investigated how much detail was resolvable during the early induction stages by testing the acuity of blindfolded sighted, naïve vOICe users. Initial performance was well above chance. Participants who took the test twice as a form of minimal training showed a marked improvement on the second test. Acuity was slightly but not significantly impaired when participants wore a camera and judged letter orientations “live”. A positive correlation was found between participants' musical training and their acuity. The relationship between auditory expertise via musical training and the lack of a relationship with visual imagery, suggests that early use of a SSD draws primarily on the mechanisms of the sensory modality being used rather than the one being substituted. If vision is lost, audition represents the sensory channel of highest bandwidth of those remaining. The level of acuity found here, and the fact it was achieved with very little experience in sensory substitution by naïve users is promising.

Keywords: sensory substitution, blindness, acuity, synthetic synesthesia, visual, auditory

Citation: Haigh A, Brown DJ, Meijer P and Proulx MJ (2013) How well do you see what you hear? The acuity of visual-to-auditory sensory substitution. Front. Psychol. 4:330. doi: 10.3389/fpsyg.2013.00330

Received: 31 January 2013; Accepted: 21 May 2013;
Published online: 18 June 2013.

Edited by:

Clare Jonas, University of East London, UK

Reviewed by:

Malika Auvray, LIMSI-CNRS, France
Thomas David Wright, University of Sussex, UK

Copyright © 2013 Haigh, Brown, Meijer and Proulx. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.

*Correspondence: Michael J. Proulx, Crossmodal Cognition Laboratory, Department of Psychology, University of Bath, 2 South, Bath BA2 7AY, UK e-mail: m.j.proulx@bath.ac.uk

Back to top