Challenges in Designing Smarter Mobile User Experiences

Mobile devices are increasingly dominating the digital world. In October 2014, The Independent alerted us to the fact that the number of mobile devices had already exceeded the number of humans in the world (Boren, 2014). According to GSMA Intelligence,¹ half the world’s population owns a mobile phone (Unique Mobile subscribers).

In the beginning, Internet access was established through desktop or laptop computers, but gradually online experiences are shifting to mobile devices (smartphones and tablets). Smartphones are now ubiquitous, especially in the developed world, where they have become true personal computers that we can (and do) carry everywhere. They offer an unprecedented combination of features, such as computational power, autonomy, size, and sensing capabilities, which gives them a “magical” power. Smartphones keep us connected with the rest of the world, no matter where we are or what we are doing, and they became addictive for a significant number of users.

Some people seem to be enchanted by smartphones, never putting them away, like little children do with their favorite toy. For them, the smartphone is the last thing they interact with before going to sleep and the first thing they grab in the morning. It is true that smartphones can stimulate social relationships by connecting people in remote places through text or image messages, voice communication, video conferencing, and social networks. But care must also be taken to avoid them becoming so intrusive that they start to restrict traditional interaction between humans when they are together. In the possession of so many digital devices with communication capabilities, some people tend to shut themselves away in their own digital bubbles and stay connected with the outside and distant world while disconnecting from their surrounding environment.

That is when direct human-to-human communication fades out and mediated communication moves in. Yvonne Rogers in her TED Talk² pointed out several possibilities for the use and design of digital technology to promote the connection between people when they are together and to bridge the gap between “mindless isolation” and “mindful ­connection,” which involve the sharing or joint use of devices (Rogers and Price, 2008). A good example of “mindful connection” is the OSMO³ gaming system for iPAD. It was created to encourage creative thinking and social interaction among children and combines an app with non-digital activities that involve physical objects, such as wooden puzzles and word games. OSMO encourages team effort, allowing children to collaborate and get out of their digital bubbles, as well as to combine their innovative digital devices with their traditional toys, benefiting from the advantages of both.

Research and industry are constantly finding new uses for these powerful “magical” devices. In the last few years, a panoply of applications have been created to support users in a plethora of tasks, exploring the advances and the ever-increasing capabilities of mobile devices. Most applications are devoted to make our lives easier, improving our well-being, supporting our everyday life tasks, or entertaining us. However, there are still many challenges to overcome, as we will discuss ahead.

Many applications have been designed to monitor (Han et al., 2015) and improve our health, for instance, helping us to increase our levels of physical activity (Harries et al., 2013) or supporting patients with chronic diseases (Preuveneers and Berbers, 2008). In this field, smartphones (in the near future together with emerging wearables, such as smartwatches), which we carry everywhere, are the best candidates to collect our personal data and advise or persuade us to perform certain actions at specific moments or adopt particular behaviors. Klasnja and Pratt (2012) present an overview of health mobile applications according to mobile devices’ features and design strategies.

More “outside the box” alternative solutions can be devised, such as the Imaginary Friend (Reis and Correia, 2011). The Imaginary Friend is a virtual character that walks along with the user and detects changes in his arousal through an electrodermal activity sensor attached to the user’s wrist. It collects the user’s emotions along the way and associates them with the locations where they were felt. The Imaginary Friend is projected on the floor using a pico-projector attached to the user’s backpack shoulder strap and accompanies the user everywhere. The projector is connected to a mobile phone running an interactive application that receives input from the electrodermal activity sensor. When a significant increase or decrease in the user’s arousal (change in conductance at the surface of the skin due to an external or internal stimuli experienced by the user) is detected, the Imaginary Friend asks the user which emotion he is feeling (the phone beeps and vibrates to call the user’s attention to his Imaginary Friend), allowing it to collect the user’s emotions. Although this system was not primarily targeted for health care, the concept can be adapted for user monitoring and advising purposes.

Moreover, the contextual moment whereupon a message is delivered assumes primordial relevance in the field of persuasion, when communication aims at provoking attitude or behavior changes. Cardoso and Romão (2014) have been working in this direction, presenting a framework for the development of context-aware mobile applications, focused on the detection of events in people’s daily lives and exploring the concept of kairos, a “window” of time during which action is most advantageous (Crowley and Hawhee, 2011). Intille (2004) and Fogg (2003) had highlighted the relevance of just-in-time messaging to motivate behavior changes, but the ability to accurately estimate the appropriate time to initiate a persuasive attempt is still a great challenge to overcome.

When developing mobile applications with UX in mind, yet another relevant challenge consists in ensuring that those applications, while omnipresent and knowledgeable of the user context, remain non-intrusive. How to determine the most appropriate time to provide the user with an advice or a suggestion without being intrusive or disruptive? When designing mobile applications for users’ advice and self-management, UX is affected by opportunity. A message or advice can only be effective when conveyed in an opportune moment in terms of the user’s needs (i.e., does the user need the message?) and convenience (i.e., would the message be disruptive?). These considerations are relevant for many areas besides health, such as tourism, marketing, or entertainment.

Some time ago, companies noticed that users needed innovative ways to interact with the TV and game consoles that would provide more natural or immersive experiences. New devices appeared, such as the Nintendo Wii with the revolutionary Wiimote, the Microsoft Kinect, and the Sony Playstation Eye, which transformed the users’ gaming experience from passively sitting down on the couch, watching a video game on TV and controlling it through a set of buttons in a traditional game remote control, into a more dynamic, natural, and realistic experience, where control could be performed through gestures. These devices with their plethora of sensors allow the development of new experiences not only for entertainment purposes but also applied to health and education (e.g., rehabilitation and gamification of physical activities); not only for individual use but also for collaborative and sometimes remote interaction between users. However, these new experiences require the acquisition of additional hardware and, in some cases, a complex setup.

Once more, the idea of using smartphones, as a ubiquitous device, originated the concept of second screen, which involves the use of a second electronic device by television viewers to connect and interact with the program they are watching. There are already several second-screen applications on the market that allow TV viewers to interact in real time with the broadcast they are watching. Mobile applications, such as Game of Thrones (Hachman, 2012), X-Factor,⁴ and The Million Pound Drop,⁵ which have been downloaded by millions of people worldwide, provide their users with additional show-related information, access to social networks, and interactive experiences synchronized with the program content, such as polls or quizzes. Some applications, such as Preplay⁶ (for baseball and American football), Viva Ronaldo⁷ (for soccer), and NHL Connect⁸ (for ice hockey), leverage the unpredictability of live sports and promote user engagement by encouraging them to guess what will happen next during a live TV broadcast.

But we can go even further and promote social interaction and user engagement with live TV sports broadcasts by bridging the gap between “mindless isolation” and “mindful connection.” When sports fans attend live sports events, they usually engage in social experiences with friends and other fans at the venue sharing the same affiliation. However, fans watching the same event through a live television broadcast end up not feeling as emotionally connected with the performers and other fans as they would if they were at the venue watching it together with thousands of other fans.

Aiming at enhancing spectators’ experience when watching live sports TV broadcasts, Centieiro et al. (2015) developed several concepts and created a range of second-screen mobile applications that deliver engaging social experiences involving remote fans. The authors were inspired by the fact that second-screen interaction is becoming widely adopted by TV viewers [a survey conducted by The Nielsen Company (2013) shows that nearly half of smartphone (46%) and tablet owners (43%) use their devices as second screens while watching TV every day] and by the users’ activities and behaviors when at the sports venue (users effusively applaud, shout, and comment with the people around them). Thus, Centieiro et al. (2015) applications allow users to interact with content that complements the TV broadcast, perform actions as if they were at the venue (e.g., applaud, clapping using the smartphone), compete with their friends, and share their feelings and comments whether they are home alone or watching the broadcast together with their friends in a public or private space. User-generated content may appear either on the TV display or on the mobile screen upon user decision.

Furthermore, Centieiro et al. (2015) suggest how to apply their concepts not only to different sports but also during live TV shows and electronic sports. They also present the challenges they faced and the guidelines they followed while developing their work, which may give a considerable contribution to the development of future second-screen applications for live broadcasted events. In spite of the above developments, it is still a big challenge to truly immerse a TV viewer into the real venue environment without significant and specific hardware setups, which are not viable for the common sports fan.

Smartphones and tablets have been replacing feature phones (the traditional mobile phones) at a fast pace, but now they have a competitor: the smartwatch. While it is not foreseeable that smartwatches are set to replace smartphones, at least in the near future, they will surely force smartphones to share the users’ attention with them. They bring about new promises and challenges. How can they help us to overcome the aforementioned challenges? How will the mobile user experience cope with this new class of device? Will we use each type of mobile device for different kinds of activities? Will we stick to one of them, smartphone or smartwatch, and make full use of their possibilities? Or will the latter become a second screen for the former?

Smart is more, so developers need to smartly design UX, and users need to smartly use and enjoy their smart devices. As Rogers² stated, it is our choice to focus on the inside of our digital bubbles, looking at our phones, and having mindless interactions or to be mindful of others and the environment around us. Mobile devices can make us oblivious of others, but they can also allow us to live vicariously and experience life through the feelings or actions of others, whenever a live experience is not viable. With mobile applications, such as Periscope,⁹ smartphones may also act as windows to different worlds, connecting the users with remote, virtual or otherwise unreachable environments, or with each other.

Since smartphones (and emerging wearables) are increasingly becoming part of our lives, it is crucial to provide the users with the best experiences for a given context. This paper unfolds some developments pursuing this goal and shows that further effort is need to convert smart devices into our perfect companions in different situations. They could become smart advisers that automatically come out with adequate solutions for our current problems, turn our once passive TV-watching activity into a highly interactive and engaging experience, help us to improve our health, or simply entertain us without disconnecting us from the surrounding environment, among many other possible life enhancing features.

Author Contributions

The author confirms being the sole contributor of this work and approved it for publication.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

This work is funded by FCT/MEC NOVA LINCS PEst UID/CEC/04516/2013.

Footnotes

1. ^https://gsmaintelligence.com/
2. ^Society Minds, Technology Doesn’t, TEDx Barcelona, 2013: http://youtube/J92H-oFexYM
3. ^https://www.playosmo.com/en/
4. ^http://bit.ly/1HhkkU6
5. ^http://bit.ly/1Kd6Efa
6. ^http://app.preplaysports.com
7. ^http://www.vivaronaldo.com
8. ^http://oneupsports.com/nhl-connect/
9. ^https://www.periscope.tv/

References

Boren, Z. D. (2014). “There are officially more mobile devices than people in the world,” in The Independent. Available at: http://www.independent.co.uk/life-style/gadgets-and-tech/news/there-are-officially-more-mobile-devices-than-people-in-the-world-9780518.html

Google Scholar

Cardoso, B., and Romão, T. (2014). “Presenting EveWorks, a framework for daily life event detection,” in Proceedings of EICS’14, Rome, Italy, 17-20 June, Vol. 2014 (ACM), 289–294.

Google Scholar

Centieiro, P., Romão, T., and Dias, A. E. (2015). “Enhancing remote spectators’ experience during live sports broadcasts with second screen applications,” in More Playful User Interfaces. Interfaces that Invite Social and Physical Interaction. Springer series: Gaming Media and Social Effects, ed. A. Nijholt (Singapore: Springer), 231–261.

Google Scholar

Crowley, S., and Hawhee, D. (2011). Ancient Rhetorics for Contemporary Students, 5th Edn. Longman Publishing Group.

Google Scholar

Fogg, B. J. (2003). Persuasive Technology: Using Computers to Change What We Think and Do. Morgan Kaufmann Publishers.

Google Scholar

Hachman, M. (2012). “HBO GO iPad app gets interactive ‘Game of Thrones’ treatment,” in PCMag. Available at: http://www.pcmag.com/­article2/0,2817,2402343,00.asp

Google Scholar

Han, T., Xiao, X., Shi, L., Canny, J., and Wang, J. (2015). “Balancing accuracy and fun: designing camera based mobile games for implicit heart rate monitoring,” in Proceedings of CHI’15, 847–856.

Google Scholar

Harries, T., Eslambolchilar, P., Stride, C., Rettie, R., and Walton, S. (2013). “Walking in the wild – using an always-on smartphone application to increase physical activity,” in Proceedings of Interact’13, Vol. 8120 (LNCS), 19–36.

Google Scholar

Intille, S. (2004). Ubiquitous computing technology for just-in-time motivation of behavior change. Stud. Health Technol. Inform. 107, 1434–1437.

PubMed Abstract | Google Scholar

Klasnja, P., and Pratt, W. (2012). Healthcare in the pocket: mapping the space of mobile-phone health interventions. J. Biomed. Inform. 45, 184–198. doi: 10.1016/j.jbi.2011.08.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Preuveneers, D., and Berbers, Y. (2008). “Mobile phones assisting with health ­self-care: a Diabetes Case Study,” in Proceedings of Mobile HCI’08, 177–186. doi:10.1145/1409240.1409260

CrossRef Full Text | Google Scholar

Reis, S., and Correia, N. (2011). “An imaginary friend that connects with the user’s emotions,” in Proceedings of ACE’11 (ACM). Article 1. New York.

Google Scholar

Rogers, Y., and Price, S. (2008). The role of mobile devices in facilitating ­collaborative inquiry in situ. Res. Pract. Technol. Enhanc. Learn. 3, 209–229.

Google Scholar

The Nielsen Company. (2013). Action Figures: How Second Screens are Transforming TV Viewing. Available at: http://bit.ly/1grK4R3

Google Scholar