Edited by: Tsutomu Fujinami, Japan Advanced Institute of Science and Technology, Japan
Reviewed by: Marco Fyfe Pietro Gillies, Goldsmiths, University of London, UK; Ian Oakley, Ulsan National Institute of Science and Technology, South Korea
*Correspondence: Junya Nakanishi
This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
In this paper, we propose the introduction of human-like communication media as a proxy for teachers to support the listening of children in school education. Three case studies are presented on storytime fieldwork for children using our huggable communication medium called Hugvie, through which children are encouraged to concentrate on listening by intimate interaction between children and storytellers. We investigate the effect of Hugvie on children's listening and how they and their teachers react to it through observations and interviews. Our results suggest that Hugvie increased the number of children who concentrated on listening to a story and was welcomed by almost all the children and educators. We also discuss improvement and research issues to introduce huggable communication media into classrooms, potential applications, and their contributions to other education situations through improved listening.
Communication with others is an important process for acquiring generic knowledge in society, such as language, communication skills, and social manners. After learners receive and interpret the information presented by caregivers or teachers, they sometimes acquire new knowledge and skills based on feedback. Obviously, a learner's ability for information comprehension is fundamental in the initial learning phase to acquire generic knowledge.
Listening is one such crucial skill, especially in school education since the information that must be learned is generally provided verbally. For example, 68% of the class time in German primary school classes and 53% in U.S. college students is spent listening (Bohlken,
Teachers and researchers have addressed the development of a curriculum for school readiness that includes listening training (Brigman and Webb,
Although they successfully provided opportunities to acquire listening skills by improving the external conditions of classrooms, they do not help students prepare their own internal states for listening. Human mental states are important in the educational curriculum for readiness to learn, including listening (Raver and Knitzer,
In this context, we focus on social interactions where people touch each other, such as a caregiver holding a child and reading a story with a picture book to her/him. Such interactions have two advantages for encouraging children to concentrate on listening. First is the impact of the tactile channel on stress reduction, which is one known effect of interpersonal touch (Gallace and Spence,
We introduce a human-like communication medium as a proxy for teachers to achieve intimate social interaction in classrooms and support both forming external information and preparing mental states for listening. In this study, we use a huggable communication medium called Hugvie with which users can strongly experience the presence of remote partners while hugging it (Minato et al.,
However, since this is the first study that introduces a huggable communication medium into classroom activities, it remains unclear how children and educators will react to it and whether they will accept it. In this paper, we present three case studies where we introduced Hugvie in storytime settings and observed how children react to investigate whether it improves children's listening. We also investigated its acceptability by children and storytellers because acceptability to new information systems indicates their successful introduction into our lives (Nickerson,
Hugvie, a huggable communication medium, is a human-shaped cushion (75-cm high and 600 g) that was designed as a communication device to give users a hugging experience. It is a soft cushion filled with polystyrene microbeads and covered with spandex fiber. Putting a hands-free mobile phone inside a pocket of its head enables users to talk while hugging it (Figure
We focused on storytelling as a typical activity since teachers spend more than half of their class time on verbal instruction from elementary school to college school in different countries (Janusik and Wolvin,
Figure
For investigating the impact of a huggable communication medium on children's listening and its acceptability by children and teachers, we introduced Hugvie into a storytime activity and observed the responses of children and storytellers. Storytime includes just storytelling and one with using tools such as pictures, books, and toys. We observed storytime to allow us to get much information about children's listening because they are mainly listening during storytime. We conducted a field experiment to observe the natural responses of children and teachers to Hugvie.
Thirty-three preschool children who are 5 or 6 years old participated in a storytime event. This study was approved by the ethics committee of the Advanced Telecommunications Research Institute International (Kyoto, Japan). Since the subjects were young children, we explained this study to all the parents and received informed consent from them. We received permission from the parents and the school to include the image records of the children for research purposes. The child participants were given Hugvies at the school's library and shown the correct posture for using them by a male experimenter: sitting straight and hugging their Hugvie to enable a device at its head to contact the children's own ear (Figure
Two coders who did not know the purpose of the experiment analyzed the recorded movies to identify the behavioral differences between the typical and Hugvie conditions. Since the time between the two conditions was different, we used the first 25 min of the movies in each condition. Children sometimes moved beyond the video camera or overlapped with another child since they were more active than we expected in the typical condition. We eliminated their data from further analysis when at least one of the coders had difficulty judging their face directions. After this preprocessing, the data collected from some children became much smaller than in the storytime because they disappeared many times from the video camera. Therefore, we used the data collected from 29 children who were observed more than the 75 percent of the whole movie in each condition for our analysis.
We defined not listening to the storytellers as children who did not direct their faces toward the storytellers as captured from the movie data. The coders coded whether each child listened to the storytellers on a second-by-second basis through the movies. The inter-coder agreement score through the used data was κ = 0.62, indicating substantial inter-observer reliability (Viera and Garrett,
Hugvie produced big changes in the children's behaviors. Figure
No children rejected or showed dislike of Hugvie, though some children did not use it.Two children who did not understand how to use it were helped by volunteers, and some near the storytellers listened directly to the story instead of through their Hugvies. They seemed to feel comfort and fun from Hugvie. For example, some children said “
The teachers and volunteers who observed the event were surprised at the result. A female volunteer said, “
We found that listening through Hugvie decreased the number of children who didn't seem to listen. While children were often distracted during typical storytime, children with Hugvie paid more attention to the storytellers. This effect appears stronger for children in the back of the room since they tend to lose focus without Hugvie due to their distance from the storyteller. On the other hand, the closer the children are to the storytellers, the weaker this effect might be since some children near the storytellers listened without their Hugvies.
Children showed no negative impressions toward Hugvie. Rather, they often expressed positive impressions such as comfort and fun. Note that the children accepted Hugvie not only in the storytime sessions by the female volunteers but also in instruction about it by a male experimenter. Perhaps, Hugvie is basically accepted by children in storytime regardless of the gender of adult storytellers.
The educators and the volunteers were also surprised at the changes in the children. This implies that the introduction of Hugvie is useful in school education. One teacher suggested that Hugvie was cast as a proxy of the storyteller: “
Hugvie showed the potential of a huggable communication medium to facilitate children's listening. However, we need more trials to test its effects because this case study is not a perfect comparison; the two environmental conditions are different. For example, storytime with Hugvie was done before the typical condition. Children might be nervous because they have few experiences of being in school, so that they might not talk and play in the former condition. Another difference is that the length of the concentration required in typical storytime is longer than with Hugvie because it is hard for children to maintain concentration for a long time. Since the rest time is also less in the typical condition than in the Hugvie condition, children might be so tired that they became easily distracted in the latter. The storytime contents were also different. For storytime without Hugvie, the volunteers often said nothing while cutting paper. Such a boring time might cause children to lose interest in the storytime. However, given the fact that volunteers with much storytime experience felt surprised by the children's behavior, Hugvie might still positively impact listening. Such surprises reflected the children's changes from more than just a few of them who didn't listen.
Practically, storytime styles vary in certain situations and such differences might change the listening support effect. For example, various persons can be storytellers. In this case study, the storytellers were mainly women with much storytime experience. Their expertise might induce Hugvie's effect. Can amateur storytellers promote the Hugvie effect? Another example is a group activity that is often performed as a class activity. While only one story was told to children at the same time in this experiment, members of different groups tell different stories to other group members in parallel. In such a situation, children have to listen to their storyteller in a noisier situation than in this experiment. Can Hugvie still support children? To address these questions and investigate how different storytime situations affect Hugvie's supportive effect, we introduced it into storytime by child storytellers as a group activity in case study 2.
To investigate whether Hugvie encourages children to concentrate on listening in such noisier situations as group activities, we introduced it into simultaneous storytime in children groups as a different storytime style from case study 1. Since most children's speaking skills are less advanced than those of adults, casting a child as the storyteller can investigate whether, regardless of a storyteller's speaking skills, Hugvie prompts listening. Additionally, we set at most four storytime groups at the same time to observe Hugvie's effect in a noisier situation. Such an investigation is valuable not only because it is the first such trial of a huggable communication medium but also because it is more difficult to pay attention to a story without Hugvie in those situations; lesser speaking skills disturb precise listening comprehension, and in simultaneous storytime groups, storyteller voices offset each other.
We introduced Hugvie into storytime sessions in the elementary school event to 139 preschool children who are 5 or 6 years old. They were divided the children into 34 groups of three to five kids with two or three 5th graders as guides of the school. Each group could freely join several sessions (including storytime) in the event. This study was approved by the ethics committee of the Advanced Telecommunications Research Institute International (Kyoto, Japan). We explained our study to all of the children's parents and received informed consent from them. In the storytime events at the school's library, they were given Hugvies and instructed how to use them by showing the correct posture as described in case study 1. We confirmed that all children could comfortably hear the experimenter's voice from their Hugvies. After that the 5th graders told stories with picture books to the preschool children for 10 min (Figure
We video-taped the storytime sessions and observed the children through the recorded movies. We received permission to include image records of the children from their parents and the school for research purposes. We categorized the children who did not direct their faces to the storytellers as children who did not listen to the story.
As the event continued, the room got louder owing to the children who were waiting to join the storytime with Hugvie. Some children chased each other around the room, and others played and/or talked with their friends or their fifth-grade guides. A few waited in silence. However, most children concentrated on the listening to the story in silence once they joined the Hugvie storytime session. Only 6% sometimes lost their attention, but they soon resumed listening without walking around or talking with others. No children rejected Hugvie. They showed such positive impressions as looking comfortable, as we observed in case study 1 when they held Hugvies.
Our results showed new potential applicable occasions for Hugvie. Regardless of the low speaking skills of the 5th grade storytellers, the preschool children listened with Hugvie. This means that anyone can be a storyteller regardless of speaking skills.
This result also suggests that Hugvie reduces not only impediments in the listening process but also the requirement needed for speaking. Although the experiment room was quite noisy due to simultaneous storytime and children who were waiting to join storytime sessions, they concentrated on listening with Hugvie. Hugvie enabled us to hold storytime in noisy environments because it produced the speaker's voice near the user's ears and relaxed the children. This achievement is completely different from what was reached by a listening support device, such as the sound field amplification system, because such devices drown out other sounds in the entire room by amplifying the speaker's voice.
We did find one negative aspect of storytime with Hugvie with respect to body posture from recorded movies. As the storytime continued, a few children showed incorrect postures, although they were correctly holding Hugvie at the beginning of the storytime: leaning on or lying astride it. While 83% held Hugvie as instructed by the experimenter, 10% leaned on Hugvie and 7% lay astride it (Figure
As with case study 1, none of the 139 children rejected Hugvie. However, this does not mean that all of the children were pleased with it. Some might have used it because the adults asked them to do so. There is room to investigate acceptability; performance may fall if children are unwilling to use a device. Since previous introductions of support devices into schools (Tanaka et al.,
For investigating how willing children are to use Hugvie, we gave them the option of using it or not in storytime after they and their parents experienced storytime with Hugvie once. Observing whether children used it in that situation shows their willingness to use it. We also asked the children about their impressions of Hugvie.
We introduced Hugvie into storytime for children at a science museum in Tokyo called Miraikan. Our participants, 29 children and their parents, were gathered in an open space of the museum by its staff members who explained the event. This study was approved by the ethics committee of the Advanced Telecommunications Research Institute International (Kyoto, Japan). We explained this study to all the parents of the subjects and received informed consent from them.
The experiments consisted of two sessions: forced and free. In the forced session, the participants were divided into two groups by families, and one group was given Hugvies and instructed how to use them. After we confirmed that all children could comfortably hear Hugvie's voice, a female volunteer with much storytime experience with children read a story with a picture book. Then we collected the Hugvies and gave them to the other group, and the volunteer told a story with an another picture book. Each storytime session lasted about 5 min.
After the forced session (storytime with parents) finished, a free session was conducted. A female staff member of the museum gathered only the children and asked them whether they wanted to use Hugvie for another storytime session (Figure
We counted the number of children who used Hugvie in the free session of each event and also checked their impressions of it in the free sessions. We collected comments from 21 parents about their impressions of their children using Hugvie. Two coders who did not know the purpose of our experiment read all the comments and categorized the parent impressions of children using Hugvie as positive, negative, or neutral. The inter-coder agreement score was κ = 0.83, indicating almost perfect inter-observer reliability (Viera and Garrett,
When we asked children whether they wanted to use Hugvie, six of 15 and seven of 14 used Hugvie in the first event. In interviewing the children in the second event, the children who were pleased with it made such comments as, “
Number of children | 13 | 16 |
Reason | Able to listen clearly | Difficult to hug and listen |
Hugvie is cute | Too noisy | |
Hugvie is not cute |
The results of the parents' impressions showed that more parents had positive impressions than negative. Twelve felt Hugvie had a positive effect on their children. Eight of 12 recognized that their children concentrated more on listening to the story with Hugvie. One mother reported that her child seemed to come back to her to be comforted during the storytime session without Hugvie. Three others hoped to use Hugvie in kindergartens or while their children were alone at home. Seven parents showed negative impressions of Hugvie. One father said he did not notice any Hugvie effect on his child. One mother found that her child looked sleepy. Two parents were worried that their children would get bored with Hugvie, and three parents wanted the interface to be improved, such as the sound quality and ease of use. The rest of the parents reported partial positive impressions of Hugvie for its usefulness for children who are far away from the storytellers although one of two coders categorized their impressions as negative or neutral.
We found some interesting behaviors of the children in the forced sessions. During storytime, nine ran up to and grabbed their parents when they were not using Hugvie, although they did not do that while using it. Two children with slightly smaller bodies than Hugvie repeatedly quit paying attention to a storyteller regardless of the conditions, and the other children almost always concentrated on listening to the storytime in both conditions.
Approximately half of the children were willing to use Hugvie, which means that a fair number of them were attracted to it after using it just once. It remains unclear whether the device's rate is high enough to introduce it into schools because no studies exist on the educational applications of similar communication devices. However, the rate is important as a baseline to improve Hugvie in respect to willingness to use it.
Our interviews and questionnaires showed that many children and their parents felt that Hugvie prompted users to concentrate on listening. In other words, Hugvie had such a strong effect that users noticed the difference caused by it. On the other hand, a few users did not feel any effect. We infer that this was mainly caused by the interface problems, including unsuitable size, sound quality, and/or ease of understanding how to use it. For example, Hugvie requires users to place their ears near the speaker because it is not very loud. Thus, misunderstanding the speaker location prevents adequate listening to the story through Hugvie, reducing its effect. These findings are important for improving Hugvie and the design policy of such support devices for telecommunication and physical interaction.
Children often run to and grab their parents, suggesting a desire to reduce their feeling of loneliness by making physical contact (Gallace and Spence,
Out of the 201 children in all the case studies, none rejected our huggable communication medium, which suggests Hugvie might be accepted by most preschool children. Yamamoto et al. reported that 2- to 3-year-old children did not exhibit positive responses to a small robot with a non-human-like appearance that showed human-like contingent actions. They argued that perhaps the children experienced the uncanny valley effect due to the conflict between the robot's appearance and its human-like actions (Yamamoto et al.,
However, not all of the children were satisfied with Hugvie. Some were unwilling to use it due to the difficulty of hugging and listening. Observations and user opinions suggested that the difficulty was caused by Hugvie's usability, including its size and stiffness, sound quality, and/or user-friendliness. This feedback provides insights into ways to improve Hugvie and highlights future research issues to be addressed before we introduce it into school education.
Case studies 2 and 3 suggested that such physical features as stiffness and size must be suitable for users. In case study 2, we found a potential problem when children use improper listening postures with Hugvie. Adult users never showed such postures since Hugvie was designed to be suitable for them. Children lean on Hugvie for support due to the immaturity of their musculoskeletal systems while adults can maintain their posture by themselves. We will verify our inferences in the future using another version of Hugvie that is stiff enough to support a child's body.
As reported in case study 3. Hugvie distracted children from listening if its size is inappropriate since small children with smaller bodies who had difficulty holding Hugvie often became distracted away from the storyteller. Another possible reason is that such distraction is caused not by a size mismatch but age. Younger children lacked the ability to sustain attention for a long time. Therefore, interesting future work might investigate the influence of size mismatch between users and Hugvie for listening with a smaller type of Hugvie.
The interviews and questionnaires of case study 3 also suggest that some users could not listen well with it because they did not understand how to use it. We need to improve Hugvie's interface to reduce such future misunderstandings. For example, marking where users should place their ears is a possible improvement. An automatic volume control system while holding Hugvie would allow each user to adjust Hugvie's volume.
As reported in case study 1, the children near the storytellers attentively listened without holding Hugvie because the storyteller's voice was louder than the sound from Hugvie. Perhaps Hugvie's voice should be the strongest stimuli among the surrounding sounds, including the storyteller's direct voice, to encourage children to use Hugvie. Although children do not need to use it when they are near a storyteller, they might benefit from using it in other aspects, as suggested in studies on interpersonal touch. For example, a brief touch from teachers motivates children to participate in lessons (Guéguen,
We also found evidence that Hugvie might benefit both teachers and children. In case study 1, as pointed out by a volunteer, the storytellers concentrated more on the story's content with Hugvie since they did not need to speak so loudly. Previous studies report that teachers often suffer from such voice problems as phonation difficulties, hoarseness, and throat pain because they have to speak loudly to control their classrooms (Yiu,
Our results also show the possibility of Hugvie's future applications. We found that our proposed storytelling system enables children to become immersed in a story even with an inexperienced storyteller in noisy environments. We also expect that Hugvie can be introduced into other activities, such as interaction with senior citizens and group work. Interaction between children and seniors is difficult because the listening skills of the latter are often poor and children's speaking skills are immature. The results of case study 2 tell us that Hugvie can deal with both problems.
Group work requires concentration on conversation among the group members. However, usually some voices are drowned out by other group conversations. Hugvie's vocal sounds can overcome the surrounding conversations without offsetting them. It can also evoke interest in a speaker (Nakanishi et al.,
Although all of our case studies suggest that a huggable communication medium has a possibility to support children's listening skill, further investigation is needed. First, we have to evaluate Hugvie's effect on children in more controlled conditions. Another important issue to be addressed is the investigation of how deeply Hugvie affects cognition. In this paper, we focused on the changes in the children's behaviors and social acceptability to Hugvie since this is the first study that introduced a huggable communication medium into educational situations. However, perhaps listening through Hugvie enhances information comprehension and memory more than usual listening. Actually, such enhancements are needed in education. Many graduate school students of college have high listening skills (McDevitt et al.,
Through three case studies, we demonstrated that huggable communication media show possibilities to encourage children to listen to others. Our huggable communication medium, Hugvie, virtually enables intimate interactions with conversation partners to improve external and internal conditions for listening. Our results showed that Hugvie, which addressed the classroom problem where children did not listen to a speaker, is accepted by children, their caregivers, and their educators. Our results also suggest that Hugvie can support communication between people who sometimes suffer from low speaking skills and low listening skills, such as children and seniors. We hope the intimate interactions mediated by huggable communication media can reduce problems of school education and other situations where listening skills are crucial and encourage people to learn from others.
JN and HS wrote the main manuscript text. All authors designed research.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. JN declare no potential conflict of interest. HS and HI are employed by ATR. ATR has patents on Hugvies. HI has consulted for Vstone Co., Ltd., which sells Hugvies, and received compensation. He also owns stock in the company.
This work was mainly supported by the Japan Science and Technology Agency (JST), the Core Research of Evolutional Science and Technology (CREST) research promotion program. Part of this work was supported by JST, the Exploratory Research for Advanced Technology (ERATO), and the ISHIGURO symbiotic human-robot interaction project. The authors would like to thank the staff at the Higashi-Hikari elementary school and Miraikan and all participants (children and their parents) for their cooperation. The authors also would like to thank the reviewers for their valuable comments and suggestions to improve the quality of the manuscript.