Sensory augmentation for the blind

Introduction

Goal-directed large-scale navigation is one of the great difficulties blind travelers face in everyday life. People with normal vision take for granted their ability to orient with the aid of distant landmarks such as large buildings, cues like street signs, and layout knowledge accessed through their visual experience of the environment. The lack of vision makes spatial orientation difficult for the blind and affects most aspects of their daily life. Without visual feedback, errors in path integration accumulate quickly. A variety of apparently simple spatial tasks, i.e., crossing a large street, put blind travelers at a severe disadvantage. Recent years have seen the development of numerous gadgets such as speaking compasses or navigational devices similar to those in cars designed to support the blind during everyday navigation. While these devices can be beneficial for orientation and navigation, they depend heavily on attentional resources so that immediate cues of the environment, i.e., approaching cars, might be neglected while attention is diverted by the navigational device. Effectively, the information overload of modern society led to a situation where more devices are available than the individual can operate simultaneously. Thus, the development of devices that can provide spatial information without drawing heavily on attentional resources could greatly enhance the life of blind travelers.

In contrast to reading off digital devices, normal perception seems effortless. Attention is used to emphasize parts of the sensory stimuli, but does not seem to be a necessary pre-condition for perception as such. Following the reasoning above, it seems desirable to supply information sampled by technical sensors in the same way as normal sensory perception is working. Indeed sensory substitution devices follow this line of thought and use a human–machine interface that provides visual information via a non-visual modality, through auditory, or tactile cues (Bach-y-Rita et al., 1969, 1998; Amedi et al., 2007; Hanneton et al., 2010). One of Bach-y-Rita et al.’s (1969) seminal experiments, a tactile vision substitution system (TVSS) for blind subjects, may serve as an example. The device converts visual input recorded by a camera into tactile signals applied to the skin of the back. Importantly, the acoustic or tactile stimulation is matched to the statistical regularities coupling action and vision. Those who trained with the device soon did not perceive the vibration on their back as the primary source of information, but rather had a qualitatively new experience, pointing toward a perceptual integration of the artificial signal. Notably, this result has only been found in subjects who were allowed to actively explore their environment with the device (Bach-y-Rita, 1972, 2004). This is fully compatible with the framework of sensorimotor contingencies (O’Regan and Noë, 2001). The framework proposes that perception is based on the knowledge of lawful relations of actions and the sensory stimulation resulting thereof. The underlying statistical regularities have to be learned through interaction with the environment. The results of such experiments with sensory substitution support the view that sensory substitution is not a passive process but has to be learned much like a new skill by interaction with the environment. The successful integration of sensory substitution devices in everyday behavior depends on learning a complex set of new perceptual skills, especially in regard to how actions like head-movements are tied to the sensory input associated with them. Development of devices supporting large-scale navigation based on the principles of sensory substitution appears a promising approach.

The concept of sensory substitution relates to enacted theories of consciousness. These predict that perception results from processes of the body that is interacting with the environment (for the philosophical and scientific ramifications, see Dewey, 1896; Merleau-Ponty, 1962; Gibson, 1979; Varela et al., 1991; Clark, 1997; Noë, 2004; Thompson, 2007; Engel, 2010). Based on these predictions, Nagel et al. (2005) designed a belt as an artificial means to supply information of an entirely new domain. The device constantly gives its bearer information about magnetic north by vibrations received on the waist. To the best of our knowledge, humans have no natural sensory organ that provides this kind of directional information. Thus, the belt provides additional information following qualitatively different transformation laws. Taking literally, it implements not sensory substitution, but sensory augmentation. In those studies the belt has been used to test predictions of enacted theories of consciousness.

Nagel et al. (2005) investigated the degree of how the new set of sensorimotor contingencies is integrated into human behavior and formalized this in a set of hypotheses: weak integration: the tactile signal provides useful information about the environment, and the integration of the signal improves performance in some behavioral tasks (in vision, this would be, i.e., simple pattern matching). Attentional resources may be used to interpret the signal. Sub-cognitive integration: the belt’s signal provides useful additional information about the environment, and the integration of the signal improves performance in some behavioral tasks. The signal can partially or completely be integrated without attentional mechanisms [in vision, letters forming words are coupled to their meaning without conscious effort after the interpretation of the letters has been learned (i.e., Stroop effect)]. Strong integration: the signal is fully integrated to a degree where the information provided is processed mandatorily. If the tactile signal is incompatible with other sensory inputs, the mismatch produces measurable results [i.e., “ventriloquism effect,” were the source of auditory information is biased toward a potential visual source (the puppet) against better knowledge (Howard and Tempelton, 1966)]. New modality: the mastery of the additional sensorimotor contingencies supplied by the belt goes beyond mere tactile perception and possibly even brings forth a new quality of sensory perception. This might be difficult to explain in words, especially to those who never experienced that quality. This set of hypotheses spans the range of reasonable degrees of integration of the new information and guides experimental tests. In a study spanning 6 weeks, subjects wore the tactile belt constantly during all waking hours to test these hypotheses. The results of the study support the hypothesis that new sensorimotor contingencies can be learned: in some (half) of the subjects, a formerly unknown perception was instilled. Additionally, their skills in different navigational tasks increased through the belt. Notably, the additional information influenced the adaptation time constant of ocular nystagmus indicating sub-cognitive processing. Thus, experiments of sensory augmentation indicate that sub-cognitive integration of simple signals is feasible, bypassing the bottleneck of attention.

Such experiments have important practical implications and may also be relevant for our understanding of the human mind. Indeed, several groups investigated sensory substitution in blind subjects, trying to re-establish a sensory modality that has been lost by utilizing another one (Bach-y-Rita et al., 1998; Arno et al., 2001; Ptito et al., 2005; Renier et al., 2005; Amedi et al., 2010). These studies show that the blind brain has the ability to meaningfully integrate visual cues delivered in a statistically meaningful way to a different modality like, i.e., audition. In Nagel et al.’s (2005) study with the tactile belt, sighted subjects could benefit from the directional information in behavioral tasks. Thus, it is possible that blind subjects could also improve their navigational skills through the directional signal. This would be a practical application for the tactile belt that has high significance. The signal given by the tactile belt could be valuable for calibration processes in navigational tasks, such as homing or pointing paradigms, but also during everyday navigation, e.g., during the exploration of new environments. Furthermore, potential benefits for self esteem and independence of blind people would be of great value. Finally, such an experiment could be interesting to further the understanding of the basic principles acting in human perceptual consciousness. Hence, testing sensory augmentation in handicapped subjects is a promising approach for the development of aids to the blind as well as for fostering our understanding of human perception.

Here we do not directly address the philosophical implications of sensory augmentation, but focus on the benefit of a sensory augmentation device for a blind subject. The subject wore the tactile belt for 6 weeks. During this period, we regularly assessed his navigational skills and conducted interviews. The tests were designed to provide evidence for three of the hypotheses (weak integration, sub-cognitive integration, new modality) formulated by Nagel et al. (2005). A test of the strong integration hypothesis would have required conditions where the information offered by the belt was incongruent with other information. In the special case of a blind subject we gave priority to the subject’s trust into the undisturbed functionality of the tactile belt and, hence, did not directly test this hypothesis. In addition to the three hypotheses stated above, we investigated the emotional impact of the belt. Thus, the study at hand examines the validity of three hypotheses: weak integration, sub-cognitive integration, and new modality. Furthermore, it investigates the questions of subjective safety and general emotional impact.

Materials and Methods

The Subject

In this case study, the subject was a male, adventitiously blind subject, age 48 years (R.B.). He suffered from retinitis pigmentosa (Hamel, 2006) and has been completely blind for 15 years. Through systematic training and preparation while he was still able to partially see, he was able to preserve an active and self-dependent lifestyle. He frequently participates in activities such as riding a tandem or guided hiking tours with other blind persons. He is right-handed, has normal hearing, a normal sense of touch, achieves normal levels in verbal and general intelligence tests, and has never suffered a neurological disease. R.B. is working as a telephone operator at a credit institute. He has never had any formal orientation and mobility training (as are often administered to the blind to teach them general way-finding skills), but studied the navigation of other blind travelers while he was still able to see. Prior to the study, he rated his sense of orientation as “very good.” For orientation in large-scale environments, he usually relies on direction estimation with the aid of the sun’s warmth. He also owns a speaking compass. R.B. wears either normal glasses or sunglasses to protect him from injuries in the eye area. When asked how he learns to navigate in an unfamiliar environment, R.B. stated that he constructs “a map in his head” onto which he adds more details as his knowledge about the environment increases. Because of his self-motivated navigational training, R.B. exhibits a high extend of understanding regarding cardinal directions. Due to his well established, cognitively grounded directional abilities, he can be seen as the other extreme in comparison to a congenitally blind subject of an earlier, similar study (Pape et al., unpublished) who had never experienced visually guided large-scale navigation and was only able to integrate the belt’s signal after an extensive explanation of cardinal directions and their meaning. R.B. agreed to take part in the study out of interest and was told only later that he would receive financial compensation for his participation. All experimental procedures were approved by the ethics-committee of the University of Osnabrück. The subject was informed of his rights prior to the study and gave written consent.

The Belt

The device for this study had to be designed in a strict and specific way to ensure its reliability and usability during the whole time of the study (6 weeks). The device was designed as a belt that delivers directional information directly to the skin by means of 30 vibration elements. The compass integrated in the device senses the direction of magnetic north. When the belt is switched on, the northernmost of the vibrating elements starts vibrating. In prior experiments we have shown that subjects feel irritated if more than one element is activated and beating rhythms are perceived (Nagel et al., 2005). Therefore, only the one element aligned closest to magnetic north is activated at a time. Attached to the belt were all necessary elements for the device’s normal functions (see Figure 1): power supply in form of three Lithium-ion polymer batteries, a digital compass, 30 pan cake vibrators (14 mm diameter, vibration frequency 170–185 Hz, vibration amplitude 1.2 G, mode of stimulation: tactile), and two control units, one to distribute the signals received from the compass and another one to distribute the power from the battery pack. The compass model was distributed from MicroStrain (Williston, VT 05495, USA) and selected because it was accurate, fast, and reliable. All necessary parts where attached to an elastic band which was wrapped in weather proof, skin friendly fabric. The belt was equipped with three battery packs that provided power for up to 20 h non-stop usage. The subject received a charging device and independently charged the batteries whenever necessary. There was no failure of the belt system or any of its parts during the time span of the experiment. The subject did not report any problems with the handling of the device; neither did he experience any negative side effects (i.e., skin irritation or adverse reaction to the vibration) from wearing the belt any time during or after the study. To investigate potential side effects of long-term tactile stimulation further studies have to be conducted. In summary, subjects wearing the belt receive directional information at all times by a vibrating signal around the waistline with minimal needs of maintenance during their whole waking time.

FIGURE 1

Figure 1. The tactile belt. (A) The belt as it was worn by the subject. A waterproof envelope covered the sensitive electronics. Inside of the envelope, parts of the batteries are visible in this photo (blue circle). (B) Inner components of the belt include two control boxes (in dark gray), the compass (red circle), and vibration elements (green circle). (C) The batteries were covered with flameproof fabric to exclude any danger for the subject and allowed continuous operation of the belt during a whole day. (D) The compass (MicroStrain, model 3DM-GX3-25) is attached to one of the control boxes. (E) The vibration elements were buttoned into the flexible band with pressure buttons that at the same time served as current conductors.

Training Schedule

The subject was asked to wear the belt during all waking hours over the entire training period of 6 weeks. He was instructed to do outdoor activities with the belt as often as possible, preferably about 90 min each day. The activities performed consisted mainly of taking walks, hiking, and being the passenger on a tandem. The subject was allowed to switch off or take off the belt during prolonged periods without trunk movements, such as sitting in front of his PC at work. The average amount of time the subject wore the belt everyday was 8 h.

We applied a series of behavioral tasks to monitor the change in the subject’s navigational abilities over the course of the study (Figure 2). A pointing task was completed five times to assess the subject’s directional estimation abilities. A homing task was done once prior to training with the belt and once after 6 weeks of training to test for improvements in small-scale path integration. The third task was based on the subject’s statement that one of the difficulties for a blind person is to walk a straight line over a longer distance, e.g., when crossing a street. The straight-line-walking-task tested the subject’s ability in that respect with and without the belt. The ability was assessed twice, once prior to the training and once after 6 weeks of training with the belt. Additionally, we conducted interviews to monitor the subject’s subjective experience with the tactile device. The subject kept a diary and answered standardized questions everyday. Every week, there was a standardized protocol of questions. Subsequently, additional questions that had come up during the week or out of the daily questionnaires were asked. Before the study started and after it ended, we conducted two longer interviews to assess the subject’s orientation abilities and gain understanding of how the blind subject usually finds his way.

FIGURE 2

Figure 2. Training Schedule. Summary of tasks for the subject during the 6-weeks of the study. Pre-training experiments were conducted prior to training with the belt, post-training experiments were conducted after the training period had ended. Four weeks after the study, the continuity of the observed effects was examined.

Experimental Tasks

To test the belt’s utility with regard to the four proposed hypotheses described above, we performed three different behavioral tasks (pointing, straight-line-walking, and homing) and conducted regular interviews. The three tasks cover a range of situations relevant for blind subjects. In the interviews, performed on a regular basis, we gathered information about the subjective experience of the subject.

Pointing

During large-scale locomotion, an understanding of the spatial environment has to be formed and updated continuously to ensure goal-directed movements. In this context, blind travelers are at a severe disadvantage toward the sighted, since they cannot correct and re-orient according to landmarks, but must form internal representations of an environment as a pre-requisite for successful navigation. Constant information about a fix point like magnetic north, relayed by the tactile belt, might facilitate the process of spatial understanding. To test whether this is the case, our subject was introduced to several environments. As a behavioral correlate of spatial learning, a pointing task was designed, since pointing accuracy is viewed as a reliable indicator of layout understanding (Golledge, 1999).

We investigate the impact of the tactile signal on spatial learning abilities such as spatial outline understanding in unfamiliar environments and the directional relations between objects. Furthermore, we test the influence of familiarity of the environment on the signal integration process in three different environments. The first environment, a public park (referred to as “the park” from this point on), was unknown to the subject prior to the study and explored with information provided by the belt. The second environment, located near a recreational area at a lake (referred to as “the lake” further on), was also unknown to the subject prior to the study, but explored only without information provided by the belt. All tasks at the lake were carried out identically to the tasks in the park. Hence, the pointing sessions at the lake serves as a control for general spatial learning ability. A third environment, the camping site where the subject regularly spends his weekends (referred to as “camping site” from this point on), was well known to the subject prior to the study and explored with information from the belt. The three environments are located in different areas of Osnabrück and have no overlapping streets. These three different environments were chosen to cover different situations to investigate the belt’s influence on spatial learning in unfamiliar and familiar surroundings.

The subject’s task was to acquaint himself with the location of six different objects in each environment. As objects, we chose landmarks typically found in natural environments (e.g., a bridge, a bank, or a pond). The accuracy of the subject’s knowledge about the environment was assessed by a pointing task. The pointing task was repeated five times over the course of the study: once prior to training with the belt, then two times during the training period in 2-week intervals, and finally twice at the end of the study (see Figure 2): first as a final measure of spatial outline learning, and afterward, in the fifth assessment of beeline estimation ability, the belt’s usage was inverted: the belt was not worn in the park and at the camping site, but was used (for the first time) at the lake. This last run served as another control of the subject’s spatial learning success.

During all experimental sessions the procedure followed a strict protocol: the experimenter led the subject to one of the six target locations in the respective environment. Different routes were chosen to approach the objects from different directions. The subject was not told which object was to be visited next. At the object location, the subject was directed to a spot that had been agreed upon during the first tour. The subject was turned into the agreed direction to make sure that his orientation toward the object was always the same, e.g., facing directly toward the object. Subsequently, the subject was given the name of another target location and asked to give a beeline estimate of the direction. The subject indicated his estimate by turning with his whole body until his nose tip was pointing toward the object in question. Additionally, the subject held a compass to determine the angular pointing direction relative to magnetic north (Figure 3A). The sequence of target locations to point to was randomly chosen by the experimenters. After pointing to one location, the subject always received affirmative, but unspecific feedback, regardless of the actual performance. This reduced the pressure on the subject performing a task where he felt handicapped. Short explanations of the different environments were given only after the study was completed. The shortest route to visit all six target locations was about 40 min at the park and at the lake and about 30 min at the camping site. The subject was introduced to all objects during the initial pre-training measurements once. During the second run that followed on the same day, the first measurements were recorded. To sum up, the pointing task required the subject to learn the location of six objects in each of three spatial environments and provided a test of spatial learning ability.

FIGURE 3

Figure 3. Pointing results. (A) To assess spatial learning ability with and without belt, the subject was introduced to six objects in three different environments. As an indicator for his spatial representations, he gave beeline direction estimations with a hand held compass. (B) Median of beeline direction estimation error. Data of all six locations were collapsed per session and environment. The median error was taken as robust indicator of pointing performance. The environments that were explored with the belts (the park and the camping site, shown in blue and gray, respectively) have lower median errors. (C) The subject’s estimates of beeline direction in degree vs. the true beeline direction in degree (dotted arrows) before and after training for two locations at the park (with belt). The angular difference (filled area) between true and estimated angular direction is the angular pointing error. The subject perceived the location on the right as “difficult” and the location on the left as “easy.” Green arrows mark the subject’s pointing estimation from the “easy” location to the “difficult” location; orange arrows mark the subject’s pointing estimation from “difficult” to “easy.” (D) At the lake (no belt): median pointing error for the six different locations. In session 1–4, the belt was not worn, and the subject pointed based on his internal estimate of the environment. In the fifth session, the subject’s belt was switched on. In the first location (in black) the belt’s information was of no use for the subject. In the subsequent locations, the subject’s median pointing error decreased. On the right is a map of the environment, including the object’s locations and the track chosen in the fifth session.

We calculated the error angle, which is defined as the absolute of the angular error between true beeline direction and the subject’s estimation of the beeline direction. The correct beeline direction between all locations was determined with the Google Inc. (2011) by drawing a straight line between the two objects and noting the angular information provided by the program. Error angles higher than 180° were mirrored by subtracting the value from 360°, since the turning direction of the subject was irrelevant.

Straight-line-walking

Keeping a straight direction while walking is a natural, effortless task for the average sighted individual, but poses enormous difficulties for a blind person. In the initial interview with the subject, he already noted that keeping the direction while crossing a large street is one of the most challenging tasks during locomotion in an urban environment for him. While tactile and auditory cues can help blind travelers to understand their immediate environment, walking straight over longer distances in a goal-directed fashion usually requires orientation with the aid of visual landmarks. Once the blind person has left the initial point and started to move, it is difficult to correct the angle without visual cues. We designed a task to assess two important questions: can the directional signal given by the belt give valuable information for a long-term stabilization of the path? And if so, are attentional resources required for the integration of the information?

We assessed the ability of the subject to walk a straight line and whether the directional signal given by the belt is helpful in such a context. A large area, about 18 m × 50 m, with no obstacles, was chosen as experimental field. To draw on the subject’s attentional resources, during part of the trials the subject was asked to subtract 7 repeatedly from 1000 and speak out loud the results. The subject’s task consisted of walking a straight line from one end of the experimental field to the other, a distance of about 50 m. The experiment consisted of four conditions: first, walking the straight line with the aid of the belt, second, walking without the belt and performing the subtractions at the same time, third, walking without the aid of the belt, and lastly walking straight with the aid of the belt and doing the subtraction exercise at the same time. There was one experimental run for each condition. Prior to the experimental runs, there were two practice runs, one with the belt and one without the belt, both without a secondary task to accustom the subject to the general setting.

The task protocol was as follows: first, the experimenter led the subject to the starting point (green circle in Figure 4) and turned the subject so that he was facing directly toward the other end of the field. When the subject was ready to go, the experimenter signaled the second experimenter who was waiting at the goal point. The second experimenter blew a whistle to give the subject an acoustic landmark for orientation to head toward. The subject was instructed to walk as quickly as possible to the other side of the area in a straight line. To give the subject a feeling of security, the first experimenter followed him directly and stopped him as soon as he crossed the limits of the experimental field. In the two trials with the subtraction exercise, the subject was told to do the calculations as quickly and accurately as possible. The angular deviation was selected as an error measure (Figure 4).

FIGURE 4

Figure 4. Results of straight-line-walking. The black line is the straight line connecting one end of the experimental field to the other. The red (without belt) and blue (with belt) lines are trajectories of the subject’s walking direction. The plus-sign denotes the trials in which the subject did an additional subtraction task. In the condition without belt, the subject had a much larger error angle when his attentional resources where occupied by the subtraction task. For all other conditions, the subject had a smaller angular error.

Homing

Homing is viewed as a tool to access the subject’s ability to do path integration based solely on idiothetic cues, i.e., based on internal representations (Nagel et al., 2005). This is opposed to allothetic cues that are generated externally, e.g., by tactile or auditory cues. In a homing task, the experimenter leads the subject on the legs of a polygon. After several turns, the experimenter steps away, and the subject returns by himself on the shortest path to the starting point of the polygon. The ability to return straight to the point of origin is found in several animal species (e.g., desert ants and geese; Mittelstaedt and Mittelstaedt, 1982; Wehner and Wehner, 1986 cited in Loomis et al., 1993), but has not been found in humans as a general occurrence. In the case of this study, the homing task was designed to test the utility of the belt in small-scale navigation and give evidence for the sub-cognitive hypothesis, since an additional memory task further increased the difficulty and drew on attentional resources. Furthermore, the homing task is a fully enacted approach to evaluate the integration of the tactile information, because the subject is responsible for choosing and following a path and finally to make a self-dependent decision on when to stop.

Homing was conducted inside a large hall where eight polygons (Figure 5C) were painted on the floor. The eight figures varied in their degree of complexity. Odd angles, sharp turns, and/or wide curves were included in some of the polygons, to test the subject’s homing ability rather than shape-recognition skills. The subject did not know any of the figures prior to the experiment. Starting at a designated starting point, the subject was led by the experimenter along the edges of the polygon toward the homing point (green circle in Figure 5C). The subject’s task was to return to the starting point in a straight line on the shortest possible way (light orange lines in Figure 5C). To test whether the subject was able to integrate the belt signal without full attentional resources, an additional memory task was conducted: before leaving the starting point, the subject was told four numbers to memorize and repeat aloud. The correctness of the numbers was assessed at the end of the trial. In the experimental condition, each of the polygons was tested twice, once with the belt and once without. The order of polygons and the belt-conditions were randomized. Homing was conducted three times: once before training with the belt, once after 6 weeks of training with the belt and once 4 weeks after the study had ended.

FIGURE 5

Figure 5. Homing results. (A) Typical homing run. The experimenter leads the subject with a wooden stick along the outside of one of the polygons. The subject wears sound canceling headphones. (B) Median of homing errors for the conditions with and without belt (blue and red respectively) for all three homing sessions (before training, after training and 4 weeks after the study). The difference between the trials with and without belt in the session directly after training is significant (p = 0.032). In the other sessions, there is no significant difference between conditions. (C) Overview of the eight different homing polygons. The subject was led along the edges of the figure starting at the gray arrow, which also indicates the direction of the run. At the green circle, the experimenter left the polygon. The subject’s task was to return straight to the starting point. The orange line indicates the optimal path back home. In the right panel, the trajectory of the actual paths chosen by the subject with belt (blue) and without belt (red) is shown with the error angles alpha and beta respectively.

The task protocol was kept as uniform as possible through all three sessions. The experiment was divided into three parts. The first two were designed to introduce the homing task to the subject; only the third part was used as experimental run. Prior to each of the three exercises, the relevant instructions were read out to the subject. The core of the three exercises was the same: the subject was led to the designated starting point by the experimenter and told that he was standing on the starting point of the figure. Subsequently, the trial commenced and the experimenter led the subject along the edges of the polygon. To avoid any bias of the subject by the experimenter through unintentional body language, the subject held on to one end of a wooden stick while the experimenter held the other end (Figure 5A). After the homing point of the figure was reached, the experimenter stopped, and slightly pushed down the wooden stick as a signal for the subject to let go of the stick. The subject independently turned as much as he deemed necessary and walked toward the chosen direction until he estimated that he had reached the starting point of the polygon. The first exercise was carried out as a practice exercise to introduce the subject to the homing task. It consisted of two simple figures (a triangle and a square). The second exercise was also for practice: the subject was led along the actual eight homing polygons, only backward, to get an estimation of the complexity of the figures. In the third run, all eight polygons were walked twice; once with the belt, once without.

Since blind people are highly sensitive to sounds and echoes from the wall and could potentially use the information gathered for navigation, the subject was given sound canceling headphones during all exercises. Due to weather conditions the sun did not shine into the hall, so the subject could not use the warmth of the sun for orientation, as he would under more favorable circumstances sometimes do outside. The subject could move freely in the hall after leaving the homing point and was only stopped if he was in danger of colliding with a wall. As strategy, we suggested to the subject to imagine that he was constantly pointing toward the starting point with his right index finger. However, the strategy was not enforced. The subject later relayed that he used the proposed strategy in a slight variation.

We measured the error angle between the shortest path from homing point to the starting point of the figure (Figure 5C, light orange) and the subject’s actual performance with and without belt (Figure 5C, rightmost panel, alpha, and beta respectively). The resulting error angles were collapsed over conditions (with belt/without belt). Turning direction did not matter, so that 180° was the largest error possible. For analysis, we applied the Wilcoxon signed rank test for paired samples and compared the distribution of error values for the trials with and without belt.

Subjective methods

A central point of interest is the qualitative change induced by the sensory augmentation device experienced by the subject; specifically with regard to whether a new quality of perception is developing. In order to address the new modality hypothesis and to find out whether the subject perceives the belt as a helpful device in everyday navigation, he completed questionnaires and was interviewed on a regular basis throughout the study, in addition to the behavioral tests.

During the study period the subject compiled a diary in the form of a standardized questionnaire (see Appendix). For this daily task he used his computer with a normal text-to-speech program. We optimized the questionnaire together with the subject to ensure usability. In addition we conducted weekly face-to-face interviews with standardized questions and open questions. The questions carefully approached the topic whether a subjective change in spatial perception or a qualitatively new experience was emerging. All standardized questions (daily and weekly) were rated on a scale from 1 to 5, where 1 was labeled “not true at all,” 2 was “a little true,” 3 was “moderately true,” 4 was “rather true,” and 5 was “exactly true.” Before the start of the training we conducted a longer interview including questions about the subject’s strategies for orientation and navigation. After the study, another longer interview assessed changes in these strategies. All face-to-face interviews were recorded and digitized. Based on the daily questionnaires and weekly interviews, the subject’s experience could be quantified on standardized scales. Please note that all quotes from the interviews as given in the results’ section have been translated from German to English. The original quotes in German can be provided on inquiry.

Results

Pointing Results

We measured the precision of the subject’s direction estimation in a pointing exercise to determine the impact of tactile signals and the familiarity of the environment on spatial learning. The absolute angular deviation in each session was collapsed over all six pointing locations to determine the median error (Figure 3B). Median pointing errors for all pointing sessions are smaller for the conditions with belt (park, camping site) than for the control condition (lake). Pointing estimates of beeline direction in the familiar environment (camping site, gray) yielded the smallest deviation between true direction and estimated direction. We applied the sign rank test for repeated measurements to the distributions of errors calculated in the different pointing sessions. In case of session 2 and session 3, the errors of the estimated airline directions from the park (with belt) and the lake (no belt) are significantly different (for session 2: p < 0.05; for session 3: p < 0.001). Collapsing the error of beeline direction estimation over all four sessions leads to a significant difference between the conditions park and lake (p < 0.001). There are no significant differences between the four sessions of either environment, i.e., the performance of the subject was stable over time. In summary, the tactile signal improved performance in an unknown environment from the first pointing session on as would be predicted by the hypothesis of weak integration.

Pointing performance suggests that the utility of the signal was partly location dependent. Two locations of the park were compared to obtain further insights. During the pointing sessions at the park, the subject indicated several times that he had more trouble with some locations than with others. In the locations perceived as “easy” by the subject, performance was high from the first session on, although the subject had only been introduced once to the six objects’ locations prior to the first airline direction estimation assessment. Performance was stable over sessions at this location, with the average pointing error staying approximately at constant level. The locations perceived as “difficult” posed a bigger problem for the subject. Performance varied over sessions and pointing estimations appeared to be more arbitrary. In Figure 3C, the estimated airline direction and the true airline direction for two object locations (one easy, one difficult) are shown before and after training. In the first pointing session before the training with the belt, the median error of the easy location was 41°, and 163° for the difficult location (in several cases the subject pointed almost in the opposite direction). In post-training session, the easy location’s median pointing error was reduced to 23° and the difficult location’s median pointing error to 84°. This leads to the conclusion that variability of performance is not on a single trial basis, but that some locations are consistently more difficult than others for the subject.

Beeline direction estimation performance is not symmetrical, e.g., pointing from location A to location B is not equal to the performance of pointing from location B to location A. The pointing error from the easy to the difficult location in the first session was 16°, in the fourth session 22°. The error in the first session from the difficult to the easy location was 172° and in the fourth session, it still came to 84°. This asymmetry indicates that the subject’s internal estimates are based on an egocentric (self-based) rather than allocentric (bird’s eye perspective) representation of the environment.

Finally, we tested the subject’s ability to instantly integrate the belt signal after the training period of 6 weeks at the lake. In all previous sessions, the subject explored the lake entirely without any information from the belt. After four sessions without the belt, the subject’s belt was switched on in a fifth session. The subject stated several times that the task of giving beeline direction estimates was subjectively facilitated through the belt. This statement was indicative of his actual performance (Figure 3D). The median pointing error for each location decreased in the fifth session, except for the location depicted in pink, where the error was always low, and the location in black. The latter location was the first on the route where the belt’s information was of no use for the subject, since knowing magnetic north did not by itself supply any new information about the other five locations. However, while walking the distance to the other locations, the subject was obviously able to adjust his internal map with the aid of the new information. His beeline direction estimations for session five show an improved understanding of the environment’s layout. After the training, the belt was instantly useful for the beeline direction estimation in an environment previously explored without the belt.

The subject’s ability to give beeline direction estimation was tested in familiar and unfamiliar environments with and without directional information supplied by the belt. The results give experimental evidence for the hypothesis of weak integration. The belt’s signal was integrated and gave the subject useful additional information about the environment: the median pointing error made at the park, where the belt was worn, was reduced by nearly 50% compared to the median pointing error made at the lake. Furthermore, the belt’s signal was instantly useful for an improvement of airline direction estimation in a semi-familiar environment, satisfying the requirements for the weak integration hypothesis: the device can instantly be used if attended to.

The subject stated in several interviews that he perceived the belt as especially useful for the correction of his internal maps of familiar environments (see Subjective Methods Results for the results of the interviews). In accordance with this statement, the median pointing error of the familiar environment “camping site” is reduced by almost 50% compared to the unfamiliar environment explored with the belt (the park). Thus, it seems that the belt’s information could be especially well integrated in a familiar environment. Since there was no control condition for the familiar environment, it is not clear whether the pointing performance was dependent on the belt or the familiarity of the environment. However, we did not want to impose yet another task on the subject, since he perceived experimental sessions as a test of his navigational abilities rather than an experiment relating the usability of the tactile belt. Furthermore, in a familiar environment it would have been difficult to ensure that he never – accidentally or not – enters the track during his daily routines while wearing the belt. Due to the lack of a suitable control condition, it cannot be finally resolved how much the belt contributes to pointing performance in a familiar environment.

There are no significant differences between the four sessions of either condition. The performance of the subject was stable and the median error of the pointing sessions remained at the same level from the first pointing session on. It is likely that the subject’s cognitive grounding in regard to cardinal directions allowed him to instantly make use of the signal. Another explanation of the results would be a difference in difficulty at the park and the lake; however, since the median pointing error at the park did not decrease even after training with the belt and the first pointing with belt at the lake yielded a reduced median pointing error compared to the four sessions without the belt, we have no indication that the difficulties of the environment were fundamentally different. The stability of the median values in all three pointing environments indicates that the subject was instantly able to integrate the belt’s signal into his behavior by concentrating on it, as is predicted by the weak integration hypothesis.

Despite the directional signal, not all locations were understood and placed into an internal map equally well. From the first pointing sessions on, the subject had a preference for certain locations that he perceived as easy, while others were perceived as difficult. His performance varied according to his own difficulty rating. It is remarkable that even after training with the belt some locations posed more difficulties than others. Since there were only six pointing locations on each track, it must remain speculative what distinguished the difficult from the easy ones. Further tests with more subjects would be necessary to determine whether the location in itself posed the difficulty or whether difficulty ratings would be depended on each subject’s personal preference. Another pointing test at the locations perceived as difficult to a wide range of objects, arranged in a circle around the location, could shed light on the contortion of the subject’s internal map. It was not possible to do further tests in the frame of this study since all testing situations imposed stress on the subject. Further tests are necessary to draw an informed conclusion.

In the fifth session, the subject received information from the belt at the lake, which in all previous sessions had been explored without directional information. The median error dropped to the level of pointing performance at the park, indicating that the subject instantly integrated the belt’s signal. The finding that the signal was useful for the re-exploration of the lake provides further evidence. All in all, the results of the beeline estimation task provide evidence for the weak integration hypothesis and can be taken as an indicator that the signal is instantly useful if attended to.

Straight-Line-Walking Task Results

We tested whether the belt enabled the subject to keep a straight direction for a longer distance and whether performance in this regard was depended on attentional mechanisms. Of the four trials, the subject had the largest deviation (15.6°) from the straight path in the dual-task condition without belt. The angular deviation in all other conditions were between 9.9° and 11.3°. The subject’s ability to walk a straight line with the aid of the belt was the same regardless of an additional task that drew on attentional capacity. In the trials without the belt, the one with an additional subtracting exercise has a larger angular error.

The task of walking a straight line is of high relevance for the blind subject. It was found that keeping a straight direction was easier for the subject if he received direction information from the tactile belt. Due to the small number of trials, no final conclusion can be drawn, but the results show a trend, indicating that the tactile belt could be a helpful device in everyday situations such as crossing a large street. Especially relevant is the finding that the integration of the belt’s signal was not disturbed by the additional subtraction exercise, suggesting that straight-line-walking could be facilitated for the blind traveler by directional information while at the same time other environmental inputs, e.g., auditory cues from traffic, could be processed in parallel. Usually, the blind are taught different methods to walk straight; however, without a directional signal they are unable to examine their own performance without further environmental cues. In terms of our hypotheses, the results of the straight-line-walking task satisfy both, the weak integration hypothesis, because the belt was helpful when attended to, and the sub-cognitive hypothesis, because the belt’s information could be integrated just as well when an additional subtraction task impedes the conscious, attentional mechanisms of signal processing. Further tests with more trials would be necessary to draw stronger conclusions. The resulting deviations from the straight direction indicate that the belt can be a helpful tool in everyday situations for blind individuals. Furthermore, the results can also be seen as further evidence for the weak integration hypothesis, i.e., that the belt’s signal can be meaningfully integrated into behavior, and the sub-cognitive hypothesis, that attention is not a necessary prerequisite for a successful integration of the signal.

Homing Results

In this part of the experiment, the subject’s ability to complete a complex path with and without information supplied by the tactile device was compared. An additional memory task restricted attentional capacity. We observed a significant difference between the distributions of error values with/without belt in the experimental session directly after training (p < 0.05, Figure 5B). Compared to the no belt condition the angular error made by the subject was significantly smaller for the trials where the subject received information from the tactile belt. In the session before training and 4 weeks after the study, no significant difference between the trials with and without belt could be determined. Since attentional capacities were partly blocked by the additional memory task, these results can be taken as evidence for a sub-cognitive integration of the signal.

The subject completed a homing task before training with the belt, directly after 6 weeks of training and 4 weeks later. In the session directly after training, the angular error was significantly decreased only in those trials where the belt was switched on. In all other sessions, no significant difference between conditions was found.

In the frame of our hypotheses, the results support the hypothesis of sub-cognitive integration of the signal. The inability of the subject to use to belt signal in the first session does not contradict the hypothesis of weak integration, since attention to the signal is one of its presumptions. Weak integration relies on conscious and attentional mechanisms, which were blocked at least partly by a memory task. Through training with the belt, sub-cognitive mechanisms in response to the signal developed and integration of the tactile information became possible for the subject in parallel to the attention demanding memory task.

Four weeks after the end of the study, the belt did not affect performance. Thus it can be presumed that the sub-cognitive mechanisms that were at work in the trial directly after training diminish after a prolonged period without training. It is possible that a longer time span of training would result in a longer storage of the newly learned sensorimotor contingencies and thus the homing results could stay stable over a longer period of time. In the present case, the signal has clearly lost its effects after 4 weeks. It seems that the hypothesis of sub-cognitive integration must be extended by a temporal component, since the sub-cognitive mechanisms have to be trained constantly to be effective.

Subjective Methods Results

The daily questionnaires assessed daily activities with the belt as well as health and mood of the subject. During working hours as a computer operator he did few trunk movements and therefore took off the belt. This resulted in 8 h per day net training time. He actively explored the belt’s function during long tandem tours with his tandem partner and by walks in familiar territory. Throughout the whole study he felt healthy at an average rating of 4.93 (scale 1–5) and slept well with an average rating of 4.00. During the study he rated his cheerfulness (average 4.65 of 5), alertness (average 4.50 of 5), quietness (average 3.15 of 5), and listlessness (average 1.23 of 5) on a daily basis. Unfortunately, the day of the final measurements of the study was an outlier in all aspects. Due to private difficulties from family matters, the subject rated his health with 4 (which is a strong statement, since he chose 5 in all other cases except one other outlier in the second week of the study), did not sleep well (2 of 5), felt neither cheerful nor alert (both rated 3 of 5), and felt more quiet (4 of 5) and listless (3 of 5) than usual. With this exception, the subject’s mood and health were stable throughout the study and the belt had no negative impact whatsoever.

Face-to-face interviews with a standardized catalog of questions were conducted weekly. Additional questions were asked in a spontaneous, situation-dependent manner. The questions of the face-to-face interviews fall into four categories: the hedonic quality of the belt; the kind of information perception; utility of the belt, and the subjective feeling of security with the belt.

Three questions targeted how the subject and those in his environment reacted to the belt. The subject rated his motivation to wear the belt as “very high” (5 of 5) throughout the study, and only in the fifth week a dip (to 4 of 5) occurred. The belt did not constrain him in his daily routine (rated 1 or 2 of 5). The only problematic issue he reported was the sweating that the belt, worn tightly around the waist, promoted during hot weather. Strangers he encountered did either not react to the belt at all, assuming it was simply a gadget for the blind, or were curious and responded positively. Overall the subject gave a strong positive evaluation of the hedonic quality of the belt.

A second focus of the questionnaire was the quality of the signal as perceived by the subject. During the first 2 weeks of training, the vibrations were prominent. The subject perceived phantom vibrations regularly after the belt was taken off in the evening. After that period, the salience of the vibration decreased. “By now, I really have to concentrate; otherwise I don’t perceive the belt, because the prickling is already being internalized.” However, the subject did not report the emergence of a qualitatively new sensation. Over the course of training the subject got accustomed more and more to the belt’s signal. In the first 3 weeks, he consciously attended to the signal to use the additional information. In the second half of the training period no conscious efforts was necessary to use the information provided by the belt.

With respect to the utility of the belt, the subject found the directional signal helpful in some situations, but less useful in others. Furthermore, his evaluation changed repeatedly during the study. Initially, he believed the belt would be of great value for navigation in unfamiliar environments. In the third week, he stated that the belt would be very useful in case he was placed in an unknown environment by himself and had to find his way back: “If I was now abandoned somewhere, I would absolutely with the belt, else I had to with the sun” (incomplete statement). However, in the following weeks the subject noticed that solely the directional information of magnetic north would not suffice for successful navigation in unknown environments. In the fifth week he contradicts his earlier statement: “If I was abandoned somewhere somehow, the belt is not really useful for me, because I don’t know the environment.”

His rating of the belt’s utility in unfamiliar environments reflects this change of mind: in the first 3 weeks, he fully agrees with the statement “I find it much easier with belt than without to orient myself in an unfamiliar environment” (rating of 5 out of 5). In the fourth week, his enthusiasm declines, and his rating drops to 2 in the fifth and sixth week. In other regards, the subject found the information received from the belt extremely useful. He fully agreed with the statement “Since I’ve started wearing the belt, I perceive the cardinal directions more consciously,” giving it a rating of 5 of 5 during the whole study. The statement “With the belt I can estimate the streets’ arrangement in respect to each other better as without belt,” initially rated with 3 of 5 in the first week, was rated with 4 or 5 of 5 from the second week on which gives an indication of the learning by the subject. The statement “With the belt, I always know where I am in relation to my home” received low ratings (in the second week 1 or 2, with one maximum of 3 of 5). However, the subject’s explanation for the low ratings is not related to the tactile device: he said repeatedly that knowledge about the direction of his home was not relevant for him as a blind person as he needs to be attentive to his immediate environment at all times. Conclusively, the subjective ratings concerning the utility of the belt showed a characteristic time course.

The subject found the belt most helpful to verify his internal representations of familiar environments. He used the expressions “mental map” or “internal map” to explain how he memorizes environments. In an unfamiliar environment, he uses an initial point for orientation and starts exploring the environment step by step. Even though he owns a speaking compass, he usually uses the warmth of the sun for directional information. In several occasions during the study, the subject expressed surprise concerning the difference of the actual direction supplied by the belt and his subjective estimation of direction, a sense he had previously believed to be very accurate. Talking about his mental map and its compatibility with the belt at the end of week three: “I have to correct it by 20°. Because I always had an estimation of direction, and now I have accurate information.” He describes how his internal map of familiar environments reluctantly changes to integrate the information supplied by the belt: “In familiar environments, the belt is useful for validation if I was right (=his mental map) […]. If the belt did not support the mental map, at first I did not want to accept it, but eventually, the belt has had me convinced.” In the last interview, the subject explained that the belt was especially helpful for an accurate understanding of familiar environments. “If a long road is slightly curved, I realized with the belt. Without the belt, I wouldn’t have realized, but the vibration moved slightly. On the camping site, we also have this long straight way that is slightly curved – I always had to count steps there, but now I don’t have to anymore. When the other vibrator vibrates, then I have to turn right after passing the forest.” In summary, the subject perceived the directional signal of limited practical use in unfamiliar environments, but experienced the tactile device as especially valuable for the correction of already existing mental maps and easing navigation.

An important part of the study investigated whether the subject would feel more secure with the belt, and whether the directional signal would introduce new behavioral possibilities for the subject. The statement “With the belt, I feel more secure in an unfamiliar environment” was rated with high agreement (4 or 5) in the first 4 weeks of the study, dropped to the lowest value in week five and climbed up to a medium rating in the last week. The statement “my ability to orient myself has subjectively improved since I’ve been wearing the belt” was given medium to high assent in the first 3 weeks, lower ratings in week four and five, and went up to high in the last week. The subject emphasized that his orientation had been already excellent prior to the study. While in the first 3 weeks he gave the statement “my ability of orientation declines after taking off the belt” a medium rating, he later on rated this statement even lower, indicating that he did not feel any subjective loss in orientation ability without the belt. Please note that this pertains to taking off the belt in the evening, not to the time after the training period. While the standardized questions regarding his subjective experience of security where rather poorly approved by the subject, he noted on several occasions that the belt had inspired him to try new things like, e.g., extending the radius of the tandem tours. In the interviews prior to the study, the subject explained that shortcuts are a risky undertaking for a blind person, since it is easy to loose the initial direction, and re-orientating is difficult once the familiar path has been left: “I can’t just walk diagonally over a meadow, I’ve tried before, if you are lucky, you make it, if not… well…[…] but for a blind person it is simply much too dangerous and unsafe. One always goes in right angles, looking for fix points… the life of a blind person is oriented in right angles.” Here, he had positive experiences with the belt: “One dares to go a little diagonal, because I can keep the direction with the aid of the vibration. I don’t digress in a curve and get lost… I would never do this otherwise (without the belt), if you get really lost as a blind person, then you don’t know where you are.” Furthermore, he explained that he feels insecure if the sun does not shine and can thus not be used for orientation during the tandem tours he regularly makes with a sighted partner. In this regard, he also perceived the belt as being useful: “It helps in the countryside, when I drive the tandem with H. To combine a familiar environment with an unfamiliar one, by simply entering the unknown environment, that’s where the belt is helpful.” In the last interview, he summarized his experiences: “I summarize: In a familiar environment, the belt provides verification, and in an unfamiliar environment, one is more courageous.” In summary, the interviews reflected an initial enthusiasm, followed by disenchantment and lastly a differentiated valuation of the belt with an emphasis on ease of navigation in familiar environments and a more courageous activity in unfamiliar environments.

Through daily and weekly interviews, we assessed different aspects of the subject’s experience during the study. The subject was highly motivated to wear the belt, especially since his mental maps of the environment were strong and well arranged before onset of the study and continued to improve with the aid of the device. He perceived the belt as helpful in situations where he extended his knowledge about familiar environments. In later weeks, he had a differentiated understanding of the belt’s utility and how the additional spatial information can be meaningfully integrated into everyday behavior.

The rating scheme through the course of the study suggests that the subject had high expectations at the beginning of the study and enthusiastically embraced the possible utilities of the directional signal. In the fourth and fifth week, his answers reflect disillusionment as he started to critically evaluate the usability of the device in his everyday life. In the last week, his ratings reflect a differentiated understanding of the belt’s potential. Notably, the subject never received quantitative feedback after taking part in the behavioral experiments and thus had no objective knowledge of his performance with the belt. The discrepancy between the subject’s experience and his real performance was particularly evident in the final interview: when asked about the homing test conducted directly after the study, he answered: “I wanted to use the belt a little, but you shouldn’t find a big difference between the trials with belt and the ones without, I believe.” This self-assessment is discordant with the behavioral results, where a significant difference was found between the trials with belt and the ones without. Thus, the subject came to a cautious rational and a positive emotional evaluation of the belt, and the latter better reflects his true performance with the belt.

Discussion

In this study, we investigated whether supplying a blind person with a directional signal for 6 weeks improves space perception and spatial learning ability, helps in everyday tasks like walking a straight line, and brings forth a perceptual change. One blind subject was provided with tactile information about magnetic north for 6 weeks through a vibrating belt. The subject’s performance in navigational tasks such as homing and pointing were significantly better when he was supplied with information from the belt, indicating improved spatial learning and path integration. In concurrent interviews, the subject stated that he felt more courageous in situations where he explored a new environment with the belt, and that he was able to correct his mental maps of familiar environments according to the relevant cardinal directions.

To what extend do these results bear on the three hypotheses regarding weak integration, sub-cognitive processing, and the induction of a new modality? How can the questions regarding a subjective overall feeling of safety and the emotional impact of the device be answered? Several results bear on the hypotheses that the information supplied by the signal could be meaningfully integrated and improved performance in navigation: in the pointing task, the belt is helpful from the first session on. In the homing task, where attention was partly distracted by an additional memory task, the subject only benefited from the belt after training with the device. We observed a comparable effect in the straight-line-walking task, where the belt improved performance only in the dual-task condition. These data give evidence for the weak integration and sub-cognitive processing hypotheses. The subject reported that he always perceived the information provided by the belt as a vibration and denied a change of spatial experience. Hence, in this blind subject we do not obtain evidence for a new perceptual modality formed by newly learned sensorimotor contingencies. Regarding the feeling of security and the emotional value of the belt, the subject summarized his experience in the final interview such that he felt more courageous in unfamiliar environments, and that he successfully used the device to re-explore and extend his knowledge about familiar environments. Nevertheless, the subject’s rational valuation of the device’s utility changed repeatedly during the study and did not match the objectively observable performance in the behavioral tests. In contrast, the emotional valuation was positive throughout the study and was a better match for the actual performance with the device. In summary, the present results give evidence for the hypotheses of weak and sub-cognitive integration, and highlight the emotional impact of the belt, but do not indicate the formation of a new modality induced by the belt.

In how far can the observations of this case report be generalized? In some tests we would have liked to conduct more trials in order to increase statistical power. However, repeatedly testing a capability (e.g., spatial navigation) where blind subjects feel handicapped is stressful. Hence, further tests and prolonged test-situations could have caused undue stress to the subject, since he perceived experimental situations as a test of his navigational abilities rather than an assessment of the belt’s utility. Moreover, although we did observe significant differences in the homing task for the specific subject, blind subjects form a highly heterogeneous group. Specifically, in comparison with an earlier study of a congenitally blind subject (Pape et al., unpublished) qualitative differences are observed (see Discussion below). Hence, the present results cannot be easily generalized to blind subjects in general. Nevertheless, with increasing interest in sensory augmentation additional data can be accumulated through combined effort and contribute to further our understanding of the heterogeneous group of blind people.

Furthermore, it seems that not all persons alike can instantly benefit from the additional directional information supplied by the belt, and that the benefits are implemented partially based on the previously existent cognitive spatial grounding. For the subject in the present study, the small amount of additional directional information and his previous approach toward navigation with the aid of cardinal directions added up. Before he participated in the study, he successfully used the sun as an external landmark and hence was able to instantly map his former knowledge to the precise information received by the belt: he was cognitively grounded in his navigational approach. A second case study with another adventitiously blind subject provides further evidence indicating the relevance of this cognitive grounding (Fenzlaff, unpublished; Hartmann, unpublished). In this case, the subject was a 58-year-old male who had lost his eyesight through an accident 15 years prior to the study. No evidence for beneficial influence of the belt on pointing performance could be demonstrated. A small difference in his homing abilities before and after the training period could be observed. His approach toward navigation was less based on cardinal directions and rather reliant on cues from his immediate environment. Thus, his skills concerning cardinal directions were not well developed. In another preceding study investigating sensory augmentation the subject was a congenitally blind male subject (Pape et al., unpublished). In this case, the belt did not improve performance in a pointing task for several weeks. However, after the subject, who had not had any prior experience with vision, was given a verbal description of the environment and thus was cognitively grounded with regard to cardinal directions it improved rapidly. In summary, spatial perception and organization prior to the study seem to play an important role for the successful integration of the vibrotactile information. Therefore, the data collected in the current study is a supplement to other studies involving sensory or cognitive grounding. The most parsimonious explanation for our results emphasizes that in the absence of visual information cognitive grounding of the new sensory signals is of central importance.

Regarding the utility of the spatial signal during everyday navigation, Nagel et al. (2005) conclude that the introduction of a new sensorimotor modality needs a behaviorally relevant domain. For the sighted in our western culture, the knowledge of the exact cardinal directions is only of marginal importance. For the blind, the signal can be an indication of unnoticed changes of directions, such as a curved road, or help with the long-term stabilization of a path during locomotion. On the one hand, modern navigational devices provide more information in this regard. It is possible for the blind today to use small navigational aids like the ones found in cars to get from one location to the other. On the other hand, the usage of these navigational devices demands attention to follow the spoken directions. In the case of the directional signal applied in this study, we provide evidence that the belt is not merely a technical device, but was used partly without attentional mechanisms. Hence, the belt might be helpful by delivering quick and precise information about the currently followed direction with only minimal disruption of attention toward the immediate environment, e.g., during crossing a road.

This study focuses on the utility of a sensory augmentation device for a blind subject. Yet it was inspired by work on the theory of sensorimotor contingencies, which we want to discuss shortly. Similar to the cognitive grounding as described above, the interaction laws with the environments must first be learned and applied so that an understanding of the ties between action and perception is implemented. The current study found no evidence concerning the implementation of a new modality through the tactile device, but that does not exclude the feasibility of such an approach. Nagel et al. (2005) reported changes in spatial perception in half of their subjects, possibly those who were previously well grounded regarding cardinal directions or those who trained most with the device. An extension of the study period could give subjects more time to actively explore the new sensorimotor contingencies. Additionally, subjects might be further motivated by realistic, adequate feedback about their performance. Thus, modifying some parameters like study length and feedback might lead to further evidence regarding the theory of sensorimotor contingencies. In studies like the present one, the experience of the subjects given particular experimental procedures cannot be satisfyingly captured by behavioral, psychophysical, or electrophysiological studies. Hence, the sensitive use of first-person data about the individual subjective experience is necessary. Studies of sensory augmentation do not only investigate subject’s performance but are equally interested in the phenomenal experience. This requires an integration of first-person data into the experimental protocol, an endeavor that is not always perceived well in the scientific community. A promising approach to a systematic use of introspective reports in studying mental processes is neurophenomenology, where neurophysiological studies are combined with formal models and first-person data (Lutz and Thompson, 2003). The aim is to generate new data by integrating phenomenological explorations into the experimental protocols and thereby adding an important dimension for understanding the experience. Regarding the current study, the acquired first-person accounts of the experience have added substantial depth to the experiment and brought up an interesting dissociation between the rational, emotional, and objectively measurable evaluation of the device.

In summary, the approach of sensory augmentation has shown here its potential to further the development of more advanced aids to the handicapped. Specifically the contrast of cautious rational assessment by the subject with high ratings in motivation and hedonic value and improved performance in the objective behavioral tests require further studies.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank the Blindenverein Osnabrück e.V. for their help in subject acquisition, Anna-Antonia Pape, Frank Schumann, and Brigitte Röder for discussion of their previous work, Alisher Numonov for technical support, and the members of the feelSpace project for their continuous support. The work was supported by the Extending Sensorimotor Contingencies to Cognition project (FP7-ICT-2009-6, Grant Agreement No.: 270212). SKN was supported by a grant from the German Ministry of Research and Education (01GP1010).

References

Amedi, A., Raz, N., Azulay, H., Malach, R., and Zohary, E. (2010). Cortical activity during tactile exploration of objects in blind and sighted humans. Restor. Neurol. Neurosci. 28, 143–156.

Pubmed Abstract | Pubmed Full Text

Amedi, A., Stern, W. M., Camprodon, J. A., Bermpohl, F., Merabet, L., Rotman, S., Hemond, C., Meijer, P., and Pascual-Leone, A. (2007). Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nat. Neurosci. 10, 687–689.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Arno, P., De Volder, A. G., Vanlierde, A., Wanet-Defalque, M. C., Streel, E., Robert, A., Sanabria-Bohórquez, S., and Veraart, C. (2001). Occipital activation by pattern recognition in the early blind using auditory substitution for vision. Neuroimage 13, 632–645.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bach-y-Rita, P. (1972). Brain Mechanisms in Sensory Substitution. New York: Academic.

Bach-y-Rita, P. (2004). Tactile sensory substitution studies. Ann. N. Y. Acad. Sci. 1013, 83–91.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bach-y-Rita, P., Collins, C. C., Sauders, F., White, B., and Scadden, L. (1969). Vision substitution by tactile image projection. Nature 21, 963–964.

CrossRef Full Text

Bach-y-Rita, P., Kaczmarek, K., Tyler, M., and Garcia-Lara, J. (1998). Form perception with a 49-point electrotactile stimulus array on the tongue. J. Rehabil. Res. Dev. 35, 427–430.

Pubmed Abstract | Pubmed Full Text

Clark, A. (1997). Being there: Putting Brain, Body, and World Together Again. Cambridge, MA: MIT Press.

Dewey, J. (1896). The reflex arc concept in psychology. Psychol. Rev. 3, 357–370.

CrossRef Full Text

Engel, A. K. (2010). “Directive minds: how dynamics shapes cognition,” in Enaction: Towards a New Paradigm for Cognitive Science, eds J. Stewart, O. Gapenne, and E. A. Di Paolo (Cambridge, MA: MIT Press), 219–243.

Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Boston: Houghton Mifflin.

Golledge, R. (1999). Wayfinding Behavior: Cognitive Mapping and Other Spatial Processes. Baltimore: JHU Press

Google Inc. (2011). Google Earth (Version 6.0.3.2197) [Software]. Available at: http://www.google.de/intl/de/earth/index.html

Hamel, C. (2006). Retinitis pigmentosa. Orphanet J. Rare Dis. 1, 40.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hanneton, S., Auvray, M., and Durette, B. (2010). The Vibe: a versatile vision-to-audition sensory substitution device. Appl. Bionics Biomech. 7, 269–276.

CrossRef Full Text

Howard, I. P., and Tempelton, W. B. (1966). Human Spatial Orientation. London: Wiley.

Loomis, J. M., Klatzky, R. L., Golledge, R. G., Cicinelli, J. G., Pellegrino, J. W., and Fry, P. A. (1993). Nonvisual navigation by blind and sighted: assessment of path integration ability. J. Exp. Psychol. Gen. 122, 73–91.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lutz, A., and Thompson, E. (2003). Neurophenomenology: integrating subjective experience and brain dynamics in the neuroscience of consciousness. J. Conscious. Stud. 10, 31–52.

Merleau-Ponty, M. (1962). Structure of Behavior. Boston: Beacon Press.

Mittelstaedt, H., and Mittelstaedt, M. L. (1982). “Homing by path integration,” in Avian Navigation, eds F. Papi, and H. G. Wallraff (New York: Springer), 290–297.

Nagel, S. K., Carl, C., Kringe, T., Märtin, R., and König, P. (2005). Beyond sensory substitution – learning the sixth sense. J. Neural Eng. 2, R13–R26.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Noë, A. (2004). Action in Perception. Cambridge, MA: MIT Press.

O’Regan, J. K., and Noë, A. (2001). A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 24, 939–973.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ptito, M., Moesgaard, S. M., Gjedde, A., and Kupers, R. (2005). Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind. Brain 128, 606–614.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Renier, L., Collignon, O., Poirier, C., Tranduy, D., Vanlierde, A., Bol, A., Veraart, C., and De Volder, A. G. (2005). Cross-modal activation of visual cortex during depth perception using auditory substitution of vision. Neuroimage 26, 573–580.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Thompson, E. (2007). Mind in Life Biology, Phenomenology and the Sciences of Mind, 1st Edn. Cambridge, MA: Belknap Press of Harvard University Press.

Varela, F., Thompson, E., and Rosch, E. (1991). The Embodied Mind. Cambridge, MA: MIT Press.

Wehner, R., and Wehner, S. (1986). Path integration in desert ants. Approaching a long-standing puzzle in insect navigation. Monitore Zoologico Italiano 20, 309–331.

Appendix

Weekly Questions

Daily Questions

Antwortmöglichkeiten kursiv

Frage 1. Hat es heute technische/praktische Probleme mit dem Gürtel gegeben? nein, ja

Falls ja, bitte beschreibe das Problem so klar wie möglich und setze Dich mit uns unmittelbar in Verbindung:

Frage 2a) Wie viele Stunden hast Du den Gürtel heute getragen? ___ Stunden

Frage 2b) Bitte besschreibe möglichst genau und vollständig, welche Aktivitäten Du heute mit dem Gürtel unternommen hast und wie lange diese jeweils dauerten.

Frage 3a) Wie lange hast Du in der vergangenen Nacht geschlafen?

Frage 3b) Hast Du heute während des Tages geschlafen? nein, ja

Frage 3c) Ich habe in der vergangenen Nacht gut geschlafen. 1(stimmt nicht), 2(stimmt wenig), 3(stimmt mittelmäßig), 4(stimmt ziemlich), 5(stimmt sehr)

Frage 3d) Hast Du in der letzten Nacht geträumt? nein, ja

Wenn ja, stellst Du eine Veränderung in deinen Träumen fest? Bitte beschreibe diese:

Frage 4a) Heute habe ich mich gesundheitlich gut gefühlt. 1(stimmt nicht), 2(stimmt wenig), 3(stimmt mittelmäßig), 4(stimmt ziemlich), 5(stimmt sehr)

Sofern Du heute gesundheitliche Einschränkungen hattest, erläutere diese bitte:

Frage 5) Bitte schätze anhand der aufgelisteten Eigenschaftswörter ein, wie Du Dich heute die meiste Zeit über gefühlt hast:

fröhlich 1(stimmt nicht), 2(stimmt wenig), 3(stimmt mittelmäßig), 4(stimmt ziemlich), 5(stimmt sehr)

wach 1(stimmt nicht), 2(stimmt wenig), 3(stimmt mittelmäßig), 4(stimmt ziemlich), 5(stimmt sehr)

ruhig 1(stimmt nicht), 2(stimmt wenig), 3(stimmt mittelmäßig), 4(stimmt ziemlich), 5(stimmt sehr)

lustlos 1(stimmt nicht), 2(stimmt wenig), 3(stimmt mittelmäßig), 4(stimmt ziemlich), 5(stimmt sehr)

Ist Dir heute während des Tragens des Gürtels etwas Besonderes aufgefallen? Wenn ja, beschreibe bitte so genau wie möglich:

Ist Dir heute während Du den Gürtel nicht getragen hast etwas Besonderes aufgefallen? Wenn ja, beschreibe bitte so genau wie möglich:

Frage 6) Freie Tagebucheintragung: